Method utilizing digital picture taking optical reader having hybrid monochrome and color image sensor

ABSTRACT

A portable hand held optical reader having a specially constructed two-dimensional image sensor array is operational in a bar code decoding mode and in a picture taking mode. The specially constructed image sensor array, in one embodiment, is a hybrid monochrome and color image sensor pixel array, wherein a first subset of the pixels are monochrome pixels devoid of wavelength selective color filter elements and a second subset of the pixels are color sensitive pixels including wavelength selective color filter elements.

CROSS REFERENCE

This application is a divisional of U.S. patent application Ser. No.11/174,447 filed Jun. 30, 2005 and titled “Digital Picture TakingOptical Reader Having Hybrid Monochrome and Color Image Sensor Array”(now U.S. Publication No. 2006/0274171) which claims right of priorityunder 35 U.S.C. §119 to Provisional Patent Application No. 60/687,606,filed Jun. 3, 2005 and titled “Digital Picture Taking Optical ReaderHaving Hybrid Monochrome And Color Image Sensor Array,” and toProvisional Patent Application No. 60/690,268, filed Jun. 14, 2005 andtitled “Digital Picture Taking Optical Reader Having Hybrid MonochromeAnd Color Image Sensor Array,” and to Provisional Patent Application No.60/692,890 filed Jun. 22, 2005 and titled “Digital Picture TakingOptical Reader Having Hybrid Monochrome And Color Image Sensor Array”and to Provisional Patent Application No. 60/694,371 filed Jun. 27, 2005and titled “Digital Picture Taking Optical Reader Having HybridMonochrome And Color Image Sensor Array.” All of the aforementionedpatent applications (U.S. patent application Ser. No. 11/174,447;Provisional Patent Application No. 60/687,606; Provisional PatentApplication No. 60/690,268; Provisional Patent Application No.60/692,890; Provisional Patent Application No. 60/694,371) are hereinincorporated by reference in their entirety.

FIELD OF THE INVENTION

The present invention relates to optical readers in general and inparticular to optical readers having picture taking functionality.

BACKGROUND OF THE PRIOR ART

Designs have been proposed for bar code decoding devices having picturetaking functionality.

In U.S. Pat. No. 6,298,176, a picture taking bar code reading device isdescribed that is equipped to output bar code data and associated imagedata. In one example described in U.S. Pat. No. 6,298,176, output imagedata is image data representing a handwritten signature. The image dataoutput by the bar code decoding device may be subject to sizecorrection, image orientation adjustment and image distortion correctionimage processing for correcting distortion resulting from an image beingcaptured at an angle.

In U.S. Publication No. US2002/0171745, a picture taking bar codereading device is described which is in communication with a remotecomputer. The bar code reading device sends image data and associatedbar code data to the remote computer. In one combined bar code/imagedata transmission scheme described in U.S. Publication No.US2002/0171745, an image data file in .PDF, .TIFF, or .BMP file formatis created at a data collection device which includes an imagerepresentation of a decoded bar code message and an image representationof the package including the bar code encoding the decoded message.

In U.S. Pat. No. 6,722,569 a picture taking bar code reading device isdescribed that includes a color image sensor and a classificationcircuit which classifies image data as being either bi-tonal image dataor color image data.

In U.S. Publication No. US2005/0001035 a picture taking bar code readingdevice is described which executes either a picture taking exposurecontrol algorithm or bar code decoding exposure control algorithmdepending on which mode is selected.

While the above references describe significant improvements in the art,there remains a need for improvement in the art of a picture takingoptical reader which is capable of picture taking functionality andexcellent bar code decoding functionality.

SUMMARY OF THE INVENTION

According to its major aspect and broadly stated, the present inventionis a picture taking optical reader having a hybrid monochrome and color(monocolor) solid state image sensor array. The hybrid image sensorarray comprises a plurality of pixels including a first subset of pixelsand a second subset of pixels, wherein the first subset of pixels aremonochrome pixels and the second subset of pixels are color sensitivepixels having wavelength selective color filter elements.

In one embodiment, the monochrome first subset of pixels is formed in acheckerboard pattern, and voids are formed at the corners of pixels ofthe first subset, such that combinations of voids of adjacent pixelsdefine open areas. Pixels of the color sensitive second subset of pixelsare formed at the open areas, and wavelength selective filter elementsare formed on pixels of the second subset but not on pixels of the firstsubset.

In another embodiment, an optical reader solid state image sensor arrayincludes a plurality of pixels formed in a plurality of rows on an ICchip in a checkerboard pattern wherein each pixel has approximately thesame dimension. The majority of pixels of the image sensor array aremonochrome pixels of the first subset. Color sensitive pixels of thesecond subset are at spaced apart positions and are uniformly orsubstantially uniformly distributed throughout the image sensor array.Color sensitive pixels may be distributed in the array in a specificpattern of uniform distribution such as a period of P=2, where everyother pixel of every other row of the image sensor array is a colorsensitive pixel, or a period of P=4 where, for every fourth row ofpixels of the array, every fourth pixel is a color sensitive pixel.

A hybrid monochrome and color sensing solid state image sensor array ofthe invention may be incorporated in an imaging module which, inaddition to having an image sensor array constructed in accordance withthe invention includes such elements as an imaging lens, an illuminationassembly including a field illumination assembly, an aiming illuminationassembly and a support member for supporting the above elements. Animaging module, in turn, may be incorporated into a hand held housingwhich encapsulates and supports the imaging assembly.

Utilizing complementary metal-oxide-silicon (CMOS) integrated circuitfabrication technologies the image sensor array in one embodiment can bemade to have selectively addressable pixels. Where the image sensorarray is constructed to have selectively addressable pixels, pixels ofthe first subset of pixels can be selectively addressed independent ofthe second subset of pixels so that image data corresponding to thefirst subset of pixels is selectively read out independent of the secondsubset of pixels. Image sensor arrays having selective read outcapability can be provided utilizing alternative fabricationtechnologies.

In a further aspect, an optical reader according to the inventionincludes separate and independently controllable reset control lines forresetting monochrome pixels and color sensitive pixels of the imagesensor array. During exposure periods for exposing color sensitivepixels, monochrome pixels may be driven into reset. During exposureperiods for exposing monochrome pixels, color sensitive pixels may bedriven into reset. Driving pixels not being selectively addressed forimage data read out into a reset state reduces cross-talk between pixelsof the image sensor array.

By incorporating within a single low cost image sensor array acombination of monochrome pixels and color sensitive pixels, an opticalreader according to the invention provides indicia decoding performanceapproximately equal to the performance of an optical reader having anall monochrome image sensor array, and picture taking performance (i.e.,the ability to obtain visual display quality color frames of image data)approximately equal to or superior to that of a digital cameraincorporating an all color pixel image sensor array, wherein each pixelof the array includes a wavelength selective filter element.

In another aspect, an optical reader image sensor array of the inventioncan include light polarizing pixels, each light polarizing pixel havinga light polarizing filter element (light polarizing filter) thatsignificantly attenuates polarized light rays generated from anappropriately polarized light source and reflected at a specular angle;thus, reducing the contribution of specularly reflected light rays togenerated image signals from the polarizing pixels. In one embodiment, afirst subset of pixels of an optical reader image sensor array aremonochrome pixels and a second subset of pixels are light polarizingpixels. For decoding decodable indicia in specular reflection readconditions, image data corresponding to the light polarizing pixels canbe selectively transferred to a decode circuit, either by way ofselecting reading out image data from the light polarizing pixels, or byselectively extracting image data corresponding to light polarizingpixels from a frame of image data including image data in addition toimage data corresponding to light polarizing pixels.

These and other aspects of the invention are described in further detailin connection with the drawings and the detailed description of theinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects and features of the invention can be better understood withreference to the drawings described below, and the claims.

FIG. 1 a is an electrical block diagram of a hand held optical reader ofthe invention including a hybrid monochrome and a color sensing solidstate image sensor array;

FIG. 1 b is a block diagram of an alternative image sensor array whichmay be incorporated into an optical reader according to the invention;

FIG. 1 c is a schematic block diagram illustrating an RF communicationscircuit according to the invention;

FIG. 1 d is a schematic block diagram illustrating a display accordingto the invention;

FIG. 1 e is a schematic view illustrating incorporation of a decodecircuit, a signature autodiscrimination circuit, a demosaicing circuit,and a fusion circuit into an optical reader according to the invention;

FIG. 2 a-2 d are various partial exploded top views of an embodiment ofa solid state image sensor array according to the invention;

FIG. 3 a is a cutaway exploded side view of a monochrome pixel accordingto one embodiment of the invention;

FIG. 3 b is a top view of the pixel shown in FIG. 3 a;

FIG. 3 c is a cutaway exploded side view of a color sensitive pixel inone embodiment of the invention;

FIG. 3 d is a top view of the pixel shown in FIG. 3 c;

FIG. 4 a is an electrical block diagram of an embodiment of an imagesensor according to the invention;

FIG. 4 b is an electrical block diagram of an image sensor array of theinvention showing incorporation of reset control lines in the imagesensor array;

FIG. 4 c is a timing diagram illustrating coordinated exposure controltiming pulses and reset control timing pulses according to theinvention;

FIGS. 5 a-5 e are various partial exploded top views of an embodiment ofa solid state image sensor array according to the invention;

FIG. 5 f is a top perspective view of an image sensor integrated circuitchip incorporating an image sensor array according to the invention withan exploded view portion illustrating a pixel pattern of color sensitive“clusters” of pixels which pattern may be distributed throughout thearray;

FIGS. 5 g-5 i are top perspective views of image sensor integratedcircuit chips incorporating a linear bar code symbol optimized imagesensor array according to the invention with respective exploded viewportions illustrating pixel patterns including “zones” of monochromepixels and “zones” of color sensitive pixels;

FIG. 5 j is a top perspective view of an image sensor integrated circuitchip incorporating a linear symbol optimized image sensor arrayaccording to the invention;

FIG. 6 a is a cutaway exploded side view of a monochrome pixel accordingto one embodiment of the invention;

FIG. 6 b is a top view of the pixel shown in FIG. 6 a;

FIG. 6 c is a cutaway exploded side view of a color sensitive pixel inone embodiment of the invention;

FIG. 6 d is a top view of the pixel shown in FIG. 6 c;

FIG. 7 a is an electrical block diagram of an embodiment of an imagesensor according to the invention;

FIG. 7 b is an electrical block diagram of an image sensor array of theinvention showing incorporation of reset control lines in the imagesensor array;

FIGS. 7 c and 7 d are schematic top views illustrating alternativeconfigurations for a reset control system including separate sets ofreset control lines for resetting a first subset of rows of pixelsindependent of resetting second subset of rows of pixels of an imagesensor array according to the invention;

FIG. 8 a is an exploded perspective view of an imaging module accordingto the invention;

FIGS. 8 b and 8 c are front and side views, respectively, of the imagingmodule shown in FIG. 8 a;

FIG. 8 d shows an illumination and aiming pattern which may be projectedby an optical reader according to the invention;

FIG. 8 e is a top view of an alternative imaging module incorporating alaser based aiming pattern generating system;

FIG. 8 f is a front view of a polarizer plate which may be included aspart of an imaging module herein, e.g., the imaging middle shown in FIG.8 a;

FIGS. 9 a and 9 b are physical form views of various hand held opticalreaders according to the invention;

FIG. 9 c is a perspective view of a hand held mobile telephone (a “cellphone”) which may incorporate a hybrid monochrome and color image sensorarray according to the invention and which may be configured accordingto the invention;

FIG. 10 is a schematic view of a system incorporating a plurality ofoptical readers according to the invention;

FIG. 11 is an application schematic view illustrating an optical readeraccording to the invention being operated to capture image datarepresenting a parcel that carries a plurality of bar code symbols;

FIG. 12 a is an application schematic view illustrating a first opticalreader according to the invention and a second remotely located opticalreader according to the invention being operated to take first andsecond digital pictures of a parcel at first and second locations thatare a distance apart for purposes of determining whether the parcel wasdamaged during delivery from the first location to the second location;

FIG. 12 b is another application schematic view illustrating an opticalreader being used to take a color picture of a delivery vehicle;

FIG. 13 a is an application schematic diagram according to the inventionillustrating an optical reader according to the invention being used toread bar codes of a vehicle and to take color pictures of a vehicle;

FIG. 13 b is a view of a VIN sticker which may be disposed on thevehicle of FIG. 13 a;

FIG. 13 c is a view of a VIN plate which may be disposed on the vehicleof FIG. 13 a;

FIG. 13 d is a view of a vehicle registration sticker which may bedisposed on the vehicle of FIG. 13 a;

FIG. 13 e is a view of an optical reader programmed to display a GUIform assisting an application wherein an optical reader, according tothe invention, is utilized to decode bar code symbols and to take colorpictures of a vehicle;

FIGS. 14 a-14 c are various flow diagrams illustrating the invention;

FIGS. 14 d-14 f are additional flow diagrams illustrating examples ofoperation of an optical reader according to the invention in an indiciadecode mode of operation;

FIGS. 14 g and 14 h are additional flow diagrams illustrating examplesof operation of an optical reader according to the invention in apicture taking mode of operation;

FIG. 14 i is a flow diagram illustrating operation of a fusion circuitof an optical reader according to the invention which processesmonochrome and color image data to produce a high resolution visualdisplay color frame of image data;

FIGS. 15 a-15 e are various image capture initiation control signaltiming diagrams illustrating the invention;

FIGS. 16 a-16 c illustrate various pixelized frames of image data whichmay be captured by an optical reader according to the invention;

FIG. 17 a is an electrical block diagram of an optical reader accordingto the invention having a plurality of imaging modules;

FIGS. 17 b and 17 c illustrate alternative hardware blocks that can beutilized with the electrical circuit of FIG. 17 a;

FIGS. 17 d and 17 e illustrate imaging modules which may be utilizedwith the reader of FIG. 17 a;

FIGS. 17 f and 17 g illustrate exemplary optical readers incorporating apair of imaging modules;

FIG. 18 a is a schematic view of a cyan-magenta-yellow (CMY) imagesensor array in accordance with the invention which may be incorporatedinto an optical reader according to the invention and which may becontrolled to generate both a decode frame of image data and a visualdisplay color frame of image data;

FIG. 19 a is a schematic view of a hybrid monochrome and polarizer imagesensor array in accordance with the invention which may be incorporatedin an optical reader according to the invention;

FIG. 19 b is a top perspective view of a hybrid monochrome and polarizerimage sensor array according to the invention with an exploded viewsection illustrating a pattern of light polarizing pixels that may bedistributed throughout the image sensor array;

FIG. 19 c is a flow diagram illustrating an exemplary operational modeof an optical reader according to the invention which incorporates ahybrid monochrome and polarizer image sensor array according to theinvention;

FIGS. 20 a and 20 b are top perspective views of a monochrome polarizerand color sensitive image sensor array according to the invention withan exploded view section illustrating a pattern of light polarizingpixels and color sensitive pixels that may be distributed throughout thearray;

FIG. 21 is a schematic view of an image sensor integrated circuit chipincorporating an image sensor array having color sensitive pixelsdisposed therein with two different periods of distribution;

FIG. 22 a is a schematic block diagram of an autodiscrimination circuitwhich may be utilized with the invention;

FIG. 22 b is a process for practicing principles of the inventionincluding automatically discriminating between different dataform types;

FIG. 22 c shows one embodiment of a plurality of curvelent detector mapswhich may be utilized with the invention;

FIG. 22 d shows another embodiment of a plurality of curvelent detectormaps which may be utilized with the invention;

FIG. 22 e is a diagrammatic representation of a histogram analysis whichmay be performed in one embodiment of the invention;

FIGS. 22 f-22 i are diagrammatic representations of an image datasegmentation process according to embodiments of the invention.

DETAILED DESCRIPTION OF THE INVENTION

An electrical block diagram of an optical reader 100 according to theinvention is shown in FIG. 1 a. Reader 100 includes a solid state imagesensor array 182A, incorporated on an image sensor integrated circuitchip 1082A shown in FIG. 1 a as a CMOS image sensor integrated circuit(IC) chip. In an important aspect, as will be described herein, imagesensor array 182A includes a plurality of pixels and wavelengthsensitive color filter elements associated with a color sensitive subsetof the pixels, wherein the remaining pixels external to the colorsensitive subset of pixels are devoid of associated wavelength selectivefilter elements. Because image sensor array 182A includes bothmonochrome pixels and color sensitive pixels, image sensor array 182Amay be termed a hybrid monochrome and color image sensor array. Reader100 further includes a processor IC chip 548 and a control circuit 552.Control circuit 552 in the embodiment of FIG. 1 a is shown as beingprovided by a central processing unit (CPU) of processor IC chip 548. Inother embodiments, control circuit 552 may be provided by e.g., aprogrammable logic function execution device such as a fieldprogrammable gate array (FPGA) or an application specific integratedcircuit (ASIC). Imaging lens 212 focuses images onto an active surfaceof image sensor array 182A and together with image sensor array 182Aforms an imaging assembly 200. Control circuit 552 executes picturetaking and indicia decoding algorithms in accordance with instructionsstored in program memory EPROM 562 which together with RAM 560 and Flashmemory 564 forms a reader memory 566. Reader memory 566 is incommunication with processor IC chip 548 via system bus 570. Mainprocessor IC chip 548 may be a multifunctional IC chip such as an XSCALEPXA25x processor IC chip including central processing unit (CPU) 552.Reader 100 further includes a field programmable gate array (FPGA) 580.Operating under the control of control circuit 552, FPGA 580 receivesdigital image data from image sensor IC chip 1082A and transfers thatimage data into RAM 560 so that the image data can be further processed(e.g., by the decoding of a bar code symbol). Processor IC chip 548 caninclude an integrated frame grabber. For example, processor IC chip 548can be an XSCALE PXA27X processor IC chip with “Quick Capture CameraInterface” available from INTEL. Where processor IC chip 548 includes anintegrated frame grabber, the integrated frame grabber may provide theframe acquisition functionality of FPGA 580. Reader 100 further includesan illumination assembly 104 and a manual trigger 216. Image sensor ICchip 1082A in the embodiment of FIG. 1 a includes an on-chipcontrol/timing circuit 1092, an on-chip gain circuit 1084, an on-chipanalog-to-digital converter 1086 and an on-chip line driver 1090. Animage sensor array which is incorporated into optical reader 100 maytake on a variety of forms. In FIG. 1 a reader 100 includes first imagesensor array 182A. However, as indicated by hardware block 208, theimage sensor array 182A may be replaced. For example, in the embodimentof FIG. 1 b, reader 100 incorporates image sensor array 182B. In otherembodiments, optical reader 100 incorporates more than one image sensorarray. Various embodiments of image sensor arrays which may beincorporated into reader 100 are described herein.

In a further aspect, reader 100 includes a radio frequency (RF)communication interface 571. Radio frequency communication interface 571may include one or more radio transceivers. Referring to the schematicdiagram of FIG. 1 c, radio frequency communication interface 571 mayinclude one or more of an 802.11 radio transceiver 5712, a Bluetoothradio transceiver 5714, a GSM/GPS radio transceiver 5716 or a WIMAX(802.16) radio transceiver 5718. Radio frequency communication interface571 facilitates wireless communication of data between device 100 and aspaced apart device 150. I/O communication interface 572 includes one ormore serial or parallel hard-wired communication interfaces facilitatingcommunication with a spaced apart device 150 as will be describedfurther in connection with FIG. 10. I/O communication interface 572 mayinclude one or more of an Ethernet communication interface, a universalserial bus (USB) interface, or an RS-232 communication interface.Optical reader 100 may further include a keyboard 508 for entering data,a pointer mover 512 for moving a pointer of a graphical user interface(GUI) and a trigger 216 for initiating bar code reading and/or picturetaking. Optical reader 100 may also include a display 504, such as amonochrome or color LED display and a touch screen 504T overlaid overdisplay 504. As shown in the schematic block diagram of FIG. 1 d,display 504 may include a display screen 5042 coupled to displaycontroller 5044 for displaying color image data. Display controller 5044receives a visual display color frame of image data from control circuit552, and reformats that data for display depending on the particularrequirements of display screen 5042, including the pixel resolution ofdisplay screen 5042. All of the components of FIG. 1 a can beencapsulated and supported by a hand held housing 101, e.g., as shown inFIGS. 9 a-9 c. Additional features and functions of the components ofreader 100 shown in FIG. 1 a are described herein.

Referring to FIG. 1 e, optical reader 100 may be regarded as havingvarious processing circuits (modules). Indicia decode circuit 1702receives image data and decodes decodable indicia therein such as barcode indicia and OCR character data. Optical reader 100 can beconfigured so that indicia decode module 1702 decodes such bar codesymbols UPC/EAN, Code 11, Code 39, Code 128, Codabar, Interleaved 2 of5, MSI, PDF417, MicroPDF417, Code 16K, Code 49, MaxiCode, Aztec, AztecMesa, Data Matrix, Qcode, QR Code, UCC Composite, Snowflake, Vericode,Dataglyphs, RSS, BC 412, Code 93, Codablock, Postnet (US), BPO4 State,Canadian 4 State, Japanese Post, KIX (Dutch Post), Planet Code and thelike, and such OCR character forms as OCR A, OCR B, and the like.Autodiscrimination circuit 1704 processes received image data anddistinguishes between handwritten character data and decodable indicia.Autodiscrimination circuit 1704 may include indicia decode circuit 1702.Autodiscrimination circuit 1704 and indicia decode circuit 1702 may bephysically embodied by a combination of control circuit 552 and memory566. Specifically, control circuit 552 operating under the control of aprogram stored in memory 562 may process image data stored in memory 560to decode decodable indicia therein or to discriminate betweenhandwritten character data and decodable indicia. Further aspects ofindicia decode circuit 1702 and autodiscrimination circuit 1704 aredescribed in copending U.S. patent application Ser. No. 10/958,779entitled, System And Method To Automatically Discriminate Between ASignature And A Barcode, filed Oct. 5, 2004 and U.S. patent applicationSer. No. 11/077,975, filed Mar. 11, 2005 entitled, Bar Code ReadingDevice With Global Electronic Shutter Control, both of which areincorporated herein by reference. As will be described further herein,optical reader 100 may further include a demosaicing circuit 1706, and afusion circuit 1708. Demosaicing circuit 1706 receives as an input acolor filter array image data frame (e.g., a Bayer pattern image) andproduces as an output a visual display color frame of image data. Fusioncircuit 1708 receives as inputs both monochrome and color image data andproduces as an output a visual display color frame of image data havinga spatial resolution at or on the order of the pixel resolution of theoptical reader's hybrid monochrome and color image sensor array. Likecircuit 1702, 1704, circuits 1706 and 1708 may be physically embodied bythe combination of control circuit 552 and memory 566. Control circuit552 as well as circuits 1702, 1704, 1706, and 1708 may be incorporatedwithin hand held housing 101 (e.g., as shown in FIGS. 9 a-9 c) or elseone or more of circuits 552, 1702, 1704, 1706, and 1708 can beincorporated in a housing of a spaced apart device 150 as described inconnection with FIG. 10.

A visual display color frame of image data as referred to herein, in oneembodiment is an image frame including a set of color indicating data ateach of a plurality of pixel positions, wherein each set of colorindicating data represents a color at a discrete position of a target1850 (shown in FIG. 8 d). Each set of color indicating data includesthree color values, e.g., a color scale value representing red, a colorscale value representing blue, and a color scale value representinggreen. Alternatively, the set of color indicating data for each pixelposition may include a cyan value, a magenta value and a valuerepresenting yellow.

In one specific example, the set of color indicating data for each pixelposition of a visual display color frame of image data output bydemosaicing circuit 1706 or fusion circuit 1708 are RGB data setsincluding 24 bits of information, wherein the first 8 bits represent ared color scale value (red value) for the pixel position, the second 8bits represent a green color scale value (green value) for the pixelposition and the third 8 bits represent a blue color scale value (bluevalue) for the pixel position.

A major feature of the invention is the construction of the opticalreader's image sensor array various embodiments of which are shown anddescribed throughout several views including the views of FIGS. 2 a-7 d.

A first embodiment of a hybrid monochrome and color sensitive(monocolor) solid state image sensor array is shown and described inFIG. 1 a and FIGS. 2 a-4 b.

Referring to FIG. 1 a and FIGS. 2 a-4 b, solid state image sensor array182A includes a monochrome first subset of pixels 250M and a colorsensitive second subset of pixels 250C. The first subset of monochromepixels 250M is formed in a checkerboard pattern and voids 253 as shownin FIG. 2 a are formed at the corners of pixels of the first subset,such that combinations of voids, e.g., voids 253-1, 253-2, 253-3, 253-4of adjacent pixels define open areas, e.g., open area 255, each openarea bounded by four pixels of the first subset. With further referenceto image sensor array 182A, pixels 250C forming a second subset ofpixels 250C are disposed in the open areas 255, and wavelength selectivefilter elements, e.g., filter element 260C, 260M, as shown in FIG. 2 bare formed on pixels of the second subset but not on pixels of the firstsubset. Monochrome pixels 250M as described herein are devoid of colorfilter elements (color filters). Pixels of the first monochrome pixelsubset are in the shape of twelve sided polygons. The pixels arecross-shaped as seen from the top view that is indicated by FIGS. 2 a-2d (the monochrome pixels are square shaped as modified by the presenceof voids 253). Pixels of the color sensitive second subset are square asseen from a top view.

In the version of image sensor array 182A shown in FIG. 2 b, colorsensitive pixels 250C of image sensor array 182A include either a cyan(Cy) filter element 260C or magenta (Mg) filter element 260M. In theversion of FIG. 2 c, color sensitive pixels 250C of image sensor array182A include either a red filter element 260R, a green filter element260G or a blue color filter element 260B (RGB filters). The colorsensitive pixels 250C can be distributed throughout image sensor array182 according to a Bayer pattern wherein there are N blue pixels, N redpixels and 2N green pixels. Color filter elements of any image sensorarray pixel as described herein can be deposited on the major body ofcolor sensitive pixels 250C by way of a depository process. As will beexplained herein, visual display color image data can be obtainedutilizing either the version of image sensor array 182A shown in FIG. 2b or the version of image sensor array 182A shown in FIG. 2 c, oranother version of image sensor array 182A such as a version includingcyan, magenta and yellow (CMY) color sensitive pixels. Because cyan andmagenta filters require only one dye and not two dyes (as in red, green,and blue filters) a version of image sensor array 182A including cyanand magenta filter elements in place of red, green and blue filterelements allows more light to pass through to a photodetector of thepixels and exhibits a higher signal to noise ratio than a versionincluding red, green and blue filters. Nevertheless, an image sensorarray having a combination of red, green and blue (RGB) filter elementsmay be preferred for certain applications. Referring to FIG. 2 d, imagesensor array 182A may include microlenses 320 for directing of lightrays incident on image sensor array 182A. Further aspects of microlenses320, including monochrome pixels, microlenses 320M, and color sensitivepixel microlenses 320C are described herein.

Exploded physical form views of an image sensor pixel array 182A, wherearray 182A is configured to operate in a global electronic shutteroperating mode are shown and described in FIGS. 3 a-3 d. A monochromepixel 250M of image sensor array 182A is shown in FIGS. 3 a and 3 b.Monochrome pixel 250M includes a photodetector 302 which may be ofphotodiode or photogate construction, a transfer gate 304, a floatingdiffusion 306, a reset transistor 307 including reset gate 308, a rowselect transistor 309 including row select gate 310 and a sourcefollower amplifier transistor 311 including amplifier gate 312. Animportant feature of pixel 250M is opaque optical shield 316. Opaqueoptical shield 316, typically comprising metal, shields light rays fromcomponents of pixel 250M other than photodetector 302. Accordingly,pixels from each of several rows of image sensor array 182A can besimultaneously exposed to light in a global electronic shutter operatingmode without the light rays modulating charges stored in floatingdiffusion 306 or another storage region. Further aspects of image sensorarrays capable of operating in a global electronic shutter operatingmode are described in U.S. patent application Ser. No. 11/077,975incorporated herein by reference. Referring to additional aspects ofpixel 250M, pixel 250M includes microlens 320 which may be disposed onlight transmissive protective layer 322. Microlens 320 collects lightfrom a larger surface area than photodetector 302 and directs lighttoward photodetector 302.

A color sensitive pixel 250C of image sensor array 182A is describedwith reference to FIGS. 3 c and 3 d. Color sensitive pixel 250C issimilar in construction to monochrome pixel 250M. Color sensitive pixel250C includes a photodetector 302 which may be of photodiode orphotogate construction, a transfer gate 304 for transferring charge fromphotodetector 250C, a floating diffusion 306, a reset transistor 307including reset gate 308, a row select transistor 309 including rowselect gate 310 and a source follower transistor amplifier 311 includingamplifier gate 312. Color sensitive pixel 250C also includes opaqueshield 320 which shields light from light sensitive components of pixel250C other than photodetector 302. Pixel 250C may also include microlens320 for increasing the amount of light incident on photodetector 302. Inaddition to the above elements color sensitive pixel 250C includes awavelength selective color filter element 260 formed thereon. Wavelengthselective color filter element 260 may be disposed intermediatemicrolens 320 and protective layer 322. In the versions of FIGS. 2 a-2d, it is seen that each color sensitive pixel 250C has four adjacentmonochrome pixels 250M.

Microlenses 320 as shown in FIGS. 3 a and 3 c are also shown in the viewof FIG. 2 d. Monochrome pixel microlens 320, 320M and color sensitivemicrolens 320, 320C may be formed on a microlens array included aplurality of microlenses. With the architecture described wherein colorsensitive pixels 250C are disposed in open areas defined by voids ofcheckerboard pattern of a first monochrome subset of pixels 250M,microlenses 320C of color sensitive pixels 250, 250C have very little(e.g., less than 3.4%) of overlap relative to microlenses 320M.

Color sensitive pixel 250C of image sensor array 182A as best seen by acomparison between FIGS. 3 b and 3 d and consumes a smaller surface areathan pixel 250M. In one version, pixel 250M includes an area, as seenfrom a top view, of about 12 nm by 12 nm while pixel 250C includes anarea, as seen from a top view, of about 6 μm by 6 μm. In anotherversion, pixel 250M includes a top surface area of about 6 μm by 6 μm,while pixel 250C includes a top surface area of about 3 μm or 3 μm. Sizereductions of pixel 250M or pixel 250, 250C may be made at low cost byreducing the number of transistors of pixel 250M and/or pixel 250C.

A transistor count of a pixel 250C of image sensor array 182A mayreadily be reduced by eliminating optically shielded floating diffusion306 in which charges are stored on a temporary basis to facilitateglobal electronic shutter operation. Accordingly, in one embodiment,monochrome pixels 250M of image sensor array 182A have more transistorsthan color sensitive pixels 250C but are capable of being exposed on aglobal electronic shutter basis, whereas color sensitive pixels 250Chave fewer transistors than monochrome pixels 250M but are not capableof being exposed on a global electronic shutter basis. In yet anotherembodiment with reference to image sensor array 182A having smallerdimensioned color sensitive pixels than monochrome pixels, therelatively larger monochrome pixels 250M have a transistor countsufficient to facilitate global shutter operation, but the relativelysmaller color sensitive pixels 250C are passive pixels requiringoff-pixel amplification, and comprise a single transistor each. Furtheraspects of global electronic shutter and rolling shutter operationsrelative to image sensor arrays which may be incorporated into reader100 are described herein.

Referring to FIG. 4 a, a high level electrical block diagram of imagesensor array 182A is shown. According to one version, image sensor array182A is an active pixel image sensor array of complementary metal oxidesemiconductor (CMOS) construction such that each pixel 250M, 250C,whether from the monochrome first subset of pixels or the colorsensitive second subset of pixels is an active pixel including a pixelamplifier 311 for amplifying signals corresponding to light incident onphotosensitive region 252. Each pixel 250M, 250C may also include anoptically shielded storage element 306. Image sensor array 182A furtherincludes two-dimensional grid of interconnects 262 which are inelectrical communication with respective column circuitry 270 and rowcircuitry 296. Row circuitry 296 and column circuitry 270 enable suchprocessing and operational tasks as selectively addressing pixels,decoding pixels, amplification of signals, analog-to-digital conversion,applying timing, read out and reset signals and the like.

Among the control lines forming interconnect grid 262 of image sensorarray 182A are pixel reset control lines. When pixels are reset byapplication of an appropriate control signal on a reset control line,residual charges which have accumulated on the pixels are connectedtemporarily to VDD so that built up charges on pixels of the imagesensor array drain out of the pixels. In accordance with the invention,image sensor array 182A includes separate reset control lines formonochrome pixels 250M and color pixels 250C. Referring to FIG. 4 b,image sensor array 182A may be constructed so that image sensor array182A has a first set of reset control lines 262R-M for resettingmonochrome pixels 250M and a second set of reset control lines 262R-Cfor resetting color pixels 250C.

In certain operating modes optical reader 100 selectively reads out awindowed frame of image data comprising image data from monochromepixels 250M. In other operating modes, optical reader 100 selectivelyreads out a windowed frame of image data comprising image data fromcolor pixels 250C. In accordance with the invention, a reset controltiming pulse can be applied to image sensor array 182A during the timethat a windowed frame of image data is being captured to reset pixels ofimage sensor array 182A that are not being selectively addressed forimage data read out. As shown by the timing diagram of FIG. 4 c, anexposure control timing pulse 354 can be coordinated with a resetcontrol timing pulse 370.

With further reference to FIG. 4 c, exposure control timing pulse 354may control exposure of monochrome pixels 250M of image sensor array182A (or alternatively, color pixels 250C) of image sensor array 182A,while reset control timing pulse 370 drives pixels not being selectivelyaddressed into a reset state. When pixels are reset, charges built up onpixels tend to be drained out of the pixels. Further, it is believedthat photons entering pixels driven into reset may be refracted so thatfewer photons become incident on neighboring pixels being exposed forimage data read out. Accordingly, coordinating the timing of an exposurecontrol pulse 354 for exposing selectively addressed pixels and a resetcontrol timing pulse 370 for resetting pixels not being selectivelyaddressed reduces cross talk between pixels.

Referring again to the view of FIG. 4 b, image sensor array 182A may beconstructed so that the presence of multiple reset control lines 162R-C,162R-M do not substantially decrease the fill factor of pixels of imagesensor array 182A. FIG. 4 b shows a schematic top view of multiple resetcontrol lines 162R-M, 162R-C incorporated in image sensor array 182,182A. According to the invention, control lines 162R-M, 162R-C can beincorporated in image sensor array 182A in a layered manner so that fora substantial portion of image sensor array 182A, control lines 164R-Mhave x, y positions that coincide with x, y positions of control line164R-C (axes are defined in FIG. 8 a). Control lines 164R-C in theembodiments of FIG. 4 b are installed at a different height (a differentZ axis position) within image sensor array 182A such that control lines162R-M and 162R-C, for substantial length of the control lines, havecommon x, y positions. Installing the multiple control lines to be ontop of one another so that the control lines have a common x, y axisposition within image sensor array 182A reduces the amount of fillfactor degradation which would otherwise result from installation of anadditional set of reset control lines within image sensor array 182A.

An alternative construction for an image sensor array according to theinvention is described with reference to FIGS. 5 a-7 b. In theembodiment of FIGS. 5 a-7 b image sensor array 182B includes a pluralityof square shaped pixels (as seen from a top view) in a checkerboardpattern, each of the pixels having substantially the same dimensions.Each pixel 250M, 250C of image sensor array 182B may be constructed tohave approximately the same top surface dimensions as seen from the topviews of FIGS. 5 a-5 i and approximately the same side viewcross-sectional dimensions as seen from the cross-sectional views ofFIGS. 6 a-6 d. Image sensor array 182B is similar to the construction ofa standard off-the-shelf monochrome image sensor array except thatselect ones of the pixels of the image sensor array have an associatedwavelength selective color filter element. Solid state image sensorarray 182B includes a plurality of pixels formed in a plurality of rows.In the version of FIGS. 5 a-5 e, a monochrome first subset of pixels250M comprise the majority of pixels of the array. Wavelength selectivecolor filter elements 260 are included in the second subset of colorsensitive pixels 250C. The color sensitive second subset of pixels 250Ccomprises pixels at spaced apart pixel positions uniformly distributedor substantially uniformly distributed throughout the plurality ofpixels forming the image sensor array 182B. In the embodiment of FIGS. 5a and 5 b, every other pixel in every other row of pixels (e.g., pixelrow 2, 4, 6 . . . ) has an associated wavelength selective color filterelement. In one example of the invention, image sensor array 182B can beprovided by including an appropriately designed color filter array on animage sensor array of an MT9M111 Digital Clarity SOC 1.3 megapixel CMOSimage sensor IC chip of the type available from Micron, Inc., an MT9V022image sensor IC chip also available from Micron, Inc. or a VV6600 1.3megapixel CMOS image sensor IC chip of the type available fromSTMicroelectronics. Other image sensor IC chips which can be utilized toprovide image sensor array 182B include MT9M413 image sensor IC chipavailable from Micron, Inc., a KAC-0311 image sensor IC chipmanufactured by Kodak, Inc. and a KAI-0340 image sensor IC chip alsomanufactured by Kodak, Inc. Operational aspects of the referencedKAI-0340 image sensor IC chip are described further herein. Variousmanufacturer product description materials respecting certain of theabove image sensor IC chips are appended to Provisional PatentApplication No. [not yet assigned] filed Jun. 22, 2005 (Express MailLabel No. EV554216715US) and Provisional Patent Application No. [not yetassigned] filed Jun. 27, 2005 (Express Mail Label No. EV554216661US)which are incorporated herein by reference. The above commercially soldimage sensor IC chips can be utilized (with additions or replacements offilter elements as are necessary) to provide any one of image sensorarrays 182B, 182C, 182D, 182F, 182G, 182H described herein.

The above referenced MT9V022 and MT9M413 image sensor IC chipsmanufactured by Micron, Inc., and KAC-0311 image sensor IC chip byKodak, Inc. are CMOS image sensor IC chips that may be operated in aglobal electronic shutter mode such that all rows of pixels subject toimage data read out have common exposure periods; that is, all rows ofpixels subject to image data read out for reading out a frame of imagedata (i.e., full frame or “windowed frame”) have a common exposure starttime and a common exposure stop time. For global electronic shutteroperation, an exposure control timing pulse, as will be described hereinis applied to the image sensor array. Exposure of each row of pixelssubject to image data read out begins at the leading edge of theexposure control timing pulse and ends at the falling edge of theexposure control timing pulse. In its technical literature, Micron, Inc.uses the trademark TRUESNAP with reference to a global electronicshutter operating mode.

Referring to FIG. 5 b, wavelength selective color filter elements(filters) formed on color sensitive pixels 250, 250C may be acombination of cyan filter elements 260C and magenta color filterelements 260M. As shown in FIG. 5 a, wavelength sensitive filters ofcolor sensitive pixels 250C may also be a combination of red filterelements 260R, green filter elements 260G and blue filter elements 260B.Because cyan and magenta filters require only one dye and not two dyes(as in red green and blue filters), the version of FIG. 5 b allows morelight to pass through to a photodetector (e.g., photodetector 302 asshown in FIG. 6 c) and exhibits a higher signal to noise ratio than theembodiment of FIG. 5 b. Nevertheless, the version of FIG. 5 a may bepreferred for certain applications.

In the embodiment of FIGS. 5 a-7 d, hybrid monochrome and color imagesensor 182B can be made by including an appropriately designed colorfilter array on a commonly available, off-the-shelf image sensor arrayin a standardly known checkerboard pattern, each pixel of the arrayhaving substantially the same dimensions. A larger portion of imagesensor array 182B is shown in FIG. 5 c, where pixels designated by theletter “c” are color sensitive pixels 250C and pixels not designated bythe letter “c” are monochrome pixels 250M. In the example of FIG. 5 c,color sensitive pixels are formed on array 182B with a period of P=2,meaning the every other pixel of every other row of pixels is a colorsensitive pixel 250C. In the version of FIG. 5 d, color sensitive pixelsare formed on array 182B with a period of P=3, meaning that every thirdpixel of every third row is a color sensitive pixel 250C. In the versionof FIG. 5 e, color sensitive pixels, c, are formed with a period of P=4,meaning that every fourth pixel from every fourth row of pixels is acolor sensitive pixel 250C. In the versions of FIGS. 5 a-5 e, each colorsensitive pixel 250C has eight adjacent monochrome pixels 250M (two sideadjacent, one top adjacent, one bottom adjacent and four corneradjacent).

Additional views of image sensor array 182B including a subset ofmonochrome pixels 250M and a subset of color sensitive pixels 250C,wherein each pixel of the image sensor array has substantially equaldimensions are shown and described in connection with FIGS. 5 f-5 j.

Referring to the version of FIG. 5 f, image sensor array 182B includesthe first subset of monochrome pixels 250M and a second subset of colorsensitive pixels 250C. The color sensitive pixels 250C of image sensorarray 182B in the version of FIG. 5 f are formed in clusters such ascluster 257R, cluster 257G and cluster 257B.

Each cluster 257 in the version of FIG. 5 f includes a plurality ofpixels in successive horizontally adjacent pixel positions, such thateach pixel of the cluster is horizontally adjacent to at least one othercolor sensitive pixel. Color sensitive clusters of pixels aredistributed uniformly or substantially uniformly throughout image sensorarray 182B. Clusters may be formed in accordance with the standardizedcolor filter pattern such as an RGB Bayer pattern or acyan-magenta-yellow (CMY) pattern. Each cluster may have a plurality ofpixels with each pixel of every individual cluster having a filterelement of the same wavelength rating. In the specific version shown inFIG. 5 f, clusters are distributed throughout image sensor array 182B ina pattern that is accordance with the pattern of Bayer color filterarray.

Cluster 257G includes three horizontally adjacent green pixels. Cluster257R includes three horizontally adjacent red pixels. Cluster 257Bincludes three horizontally adjacent blue pixels. As will be describedfurther in connection with FIG. 7 c, the version of image sensor array182B including a distribution of color sensitive pixels in horizontallyarranged clusters as shown in FIG. 5 f is particularly useful where itis desired to include in image sensor array 182B separate andindependently controllable reset control lines 262R-M and 262R-C forseparately and independently resetting monochrome pixels of image sensorarray 182B and color sensitive pixels of image sensory array 182Bwithout increasing the thickness of image sensor array 182B.

Referring now to the versions of image sensor array 182B shown in FIG. 5g-5 j, image sensor array 182B having a subset of monochrome pixels in asubset of color sensitive pixels may be configured to include “zones” ofmonochrome pixels and “zones” of color sensitive pixels. A “zone” ofpixels herein is a collection of positionally related pixels at aspecified area of an image sensor array each having a color filterelement or alternatively, each being without a color element. Examplesof zones described herein comprise all pixels of one row of pixels orall pixels of each of several consecutive rows of pixels. In the versionof FIG. 5 g, image sensor array 182B includes two color sensitive zonesof pixels 2500C and a single monochrome zone of pixels 2500M. Each zoneof pixels comprises a plurality of horizontally, vertically ordiagonally adjacent pixels. The plurality of pixels of a monochrome zoneof pixels, e.g., zone 2500M are all devoid of a color sensitive filterelement. The plurality of adjacent pixels in a color sensitive zone ofpixels, e.g., zone 2500C, all include a color sensitive filter element.

Referring to the version of FIG. 5 g, monochrome zone of pixels 2500M isinterposed between a pair of color sensitive zones of pixels 2500C.Monochrome zone of pixels 2500M in the version of FIG. 5 g comprises asingle row of pixels of image sensor array 182B at or approximately thecenter of image sensor array 182B. The first color sensitive zone ofpixels of an image sensor array 182B includes all pixels from the row ofpixels of zone 2500M up to the top row of image sensor array 182B. Thesecond color sensitive zone of pixels 2500C in the version of FIG. 5 gincludes all pixels from all rows from the center row monochrome zone ofpixels 2500M down to the bottom row of pixels of image sensor array182B. The color filter elements of color sensitive pixels 250C of imagesensor array 182B may be formed in a standard color filter pattern,e.g., an RGM Bayer color filter pattern or a CMY pattern.

Referring to FIG. 5 h, another version of image sensor array 182B isshown and described. The version of FIG. 5 h is similar to the versionof FIG. 5 g except that the monochrome zone of pixels 2500M is expandedto include ten consecutive rows of pixels at the center or approximatelythe center of image sensor array 182B.

In the version of image sensor array 182B as shown in FIG. 5 i, a singlecolor sensitive zone of pixels 2500C is interposed between tworelatively small width monochrome zones of pixels 2500M formed at thetop and bottom of image sensor array 182B respectively. In the versionof image sensor array 182B shown in FIG. 5 a, the first monochrome zoneof pixels 2500M comprises the first ten rows of pixels of image sensorarray 182B and a second monochrome zone of pixels 2500M includes pixelsof the bottom ten rows of image sensor array 182B. Color sensitive zoneof pixels 2500C in the version of FIG. 5 i includes all pixels of thearray excluding the first ten and the last ten rows of pixels of imagesensor array 182B. In the versions of FIG. 5 h and FIG. 5 i, the pixelsof the color sensitive zones 2500C shown may include color filterelements in accordance with the pattern of a standardized color filterarray, e.g., an RGB Bayer pattern or a CMY pattern.

The version of image sensor array 182B shown in FIG. 5 j is similar inconstruction to the version of FIG. 5 g except that the version of FIG.5 j includes additional monochrome zones of pixels 2500M. In the versionof FIG. 5 j image sensor array 182B includes a pair of diagonal zones ofmonochrome pixels 2500M-D extending through a center (actual orapproximate) of image sensor array 182B and a vertically extending zoneof monochrome pixels 2500M-V extending through a center of image sensorarray 182B. The linear zones of monochrome pixels 2500M shown in theversion of FIG. 5 j may include a minor dimension equal to one pixelwidth or more than one pixel width. For example, the verticallyextending monochrome zone of pixels 2500M of FIG. 5 j may include pixelpositions of one column of pixels or of a plurality of columns ofpixels. Likewise, the diagonally extending linear monochrome zones ofpixels 2500M of FIG. 5 g may include pixel positions of a singlediagonal row of pixels or alternatively, of a plurality of diagonal rowsof pixels.

It will be seen that the versions of image sensor array 182B shown inFIGS. 5 g-5 j are particularly well suited for use in picture takingoptical readers which in bar code decoding applications are expected todecode linear bar code symbols. The image sensor arrays of FIGS. 5 g-5 jmay be referred to as linear symbol optimized image sensor arrays. Aswill be described in further detail herein, image data corresponding tomonochrome zones of pixels 2500M in the versions of FIGS. 5 g-5 j can beselectively addressed and read out independently of image data from rowsfrom color sensitive zones 2500C of pixels. In bar code decodingapplications, control circuit 552 may selectively address pixels ofmonochrome zones 2500M and read out image data from monochrome zones ofpixels 2500M as shown in FIGS. 5 g-5 i and transfer such image data toindicia decode circuit 1702 for decoding of a linear bar code symbol.For picture taking applications, control circuit 552 may selectivelyaddress pixels of a color sensitive zone or zones of pixels 2500C andselectively read out image data from color sensitive zone or zones 2500Cand process such color image data into a visual display color frame ofimage data. The processing as will be explained further herein mayinclude such steps as executing a demosaicing routine to convert colorfilter pattern image data into a visual display format and interpolationof color pixel values corresponding to the missing pixel positions atthe pixel positions occupied by a monochrome zone or zones 2500M ofpixels.

In FIGS. 6 a-6 d, exploded physical form view of pixels of image sensorarray 182, 182B are shown. A monochrome pixel 250M of image sensor array182B is shown in FIGS. 6 a and 6 b. Pixel 250M includes a photodetector302 which may be of photodiode or photogate construction, a transfergate 304, a floating diffusion 306, a reset transistor 307 includingreset gate 308, a row select transistor 309 including row select gate310 and a source follower amplifier transistor 311 including amplifiergate 312. An important feature of pixel 250M is opaque optical shield316. Opaque optical shield 316, typically comprising metal, shieldslight rays from components of pixel 250M other than photodetector 302.Accordingly, pixels from each of several rows of image sensor array 182Acan be simultaneously exposed to light in a global electronic shutteroperating mode without the light rays modulating charges stored infloating diffusion 306 or another storage region. Further aspects ofimage sensor arrays capable of operating in a global electronic shutteroperating mode are described in U.S. patent application Ser. No.11/077,975 incorporated herein by reference. Referring to additionalaspects of pixel 250M, pixel 250M includes microlens 320 which may bedisposed on light transmissive protective layer 322. Microlens 320collects light from a larger surface area than photodetector 302 anddirects light toward photodetector 302.

A color sensitive pixel 250C of image sensor array 182B is describedwith reference to FIGS. 6 c and 6 d. Color sensitive pixel 250C issimilar in construction to monochrome pixel 250M. Color sensitive pixel250C includes a photodetector 302 which may be of photodiode orphotogate construction, a transfer gate 304 for transferring charge fromphotodetector 250C, a floating diffusion 306, a reset transistor 307including reset gate 308, a row select transistor 309 including rowselect gate 310 and a source follower transistor amplifier 311 includingamplifier gate 312. Color sensitive pixel 250C also includes opaqueshield 320 which shields light from light sensitive components of pixel250C other than photodetector 302. Pixel 250C may also include microlens320 for increasing the amount of light incident on photodetector 302. Inaddition to the above elements, color sensitive pixel 250C includes awavelength selective color filter element 260 formed thereon. Wavelengthselective color filter element 260 may be disposed intermediatemicrolens 320 and protective layer 322.

A high level electrical block diagram of image sensor array 182B isshown in FIG. 7 a. Image sensor array 182B may be of CMOS constructionand may be an active pixel image sensor array such that each pixel 250of image sensor array 182B includes a pixel amplifier 311. Each pixel250 of image sensor array may further have a photosensitive region 252and an optically shielded storage element 306. Image sensor array 182Bfurther includes a two-dimensional grid of interconnects 262 which arein electrical communication with respective column circuitry 270 and rowcircuitry 296. Row circuitry 296 and column circuitry 270 enable suchprocessing and operational tasks as selectively addressing pixels,decoding pixels, amplification of signals, analog-to-digital conversion,and applying timing, read out and reset signals.

Reset control lines of interconnect grid 262 are shown in FIG. 7 b.Image sensor array 182B may have multiple sets of reset control lines sothat monochrome pixels 250M of image sensor array 182B can be resetindependently of color sensitive pixels 250C of image sensor array 182Bas described previously in connection with the description of imagesensor array 182B. According to the invention, control lines 262R-M,262R-C can be incorporated in image sensor array 182B in a layeredmanner so that for a substantial portion of image sensor array 182B,control lines 262R-M have x, y positions that coincide with x, ypositions of control line 262R-C (axes are defined in FIG. 8 a). Controllines 262R-C in the embodiment of FIG. 7 b are installed at a differentheight (a different Z axis position) within image sensor array 182Brelative to control lines 262R-C such that control lines 262R-M and262R-C, for substantial length of the control lines, have common x, ypositions. Installing the multiple control lines to be on top of oneanother so that the control lines have a common x, y axis positionwithin image sensor array 182B reduces the amount of fill factordegradation which would otherwise result from installation of anadditional set of reset control lines within image sensor array 182B.

Referring to FIGS. 7 c and 7 d, image sensor array 182B may beconfigured to include separate and independent reset control lines forseparately and independently resetting monochrome pixels 250M and colorsensitive pixels 250C without increasing the overall thickness of imagesensor array 182B. While disposing reset control lines on top of oneanother as described in connection with FIGS. 4 b and 7 b providessignificant advantages; the image sensor array is made thicker with sucharrangement which adds to manufacturing costs. Referring to FIG. 7 c, aversion of image sensor array 182B is illustrated having a first set ofreset control lines 262, 262R-M for resetting monochrome pixels 250M anda second set of reset control lines 262, 262R-C for resetting colorsensitive pixels 250C of image sensor array 182B. The reset control lineconfiguration of FIG. 7 c may be utilized with the color sensitive pixeldistribution shown in FIG. 5 f to provide an image sensor array 182Bhaving separate and independently controllable reset control lines forseparately resetting monochrome pixels 250M and color sensitive pixels250C and which exhibits a thickness equal to a thickness of a commonlyavailable off-the-shelf image sensor array. In the version of imagesensor array 182B shown in FIG. 7 c, the reset control lines ofmonochrome pixel rows are electrically connected together and the resetcontrol lines of rows of pixels including color sensitive pixels areelectrically connected together. The commonly connected reset controllines of the monochrome pixel rows are designated with the referencenumeral 262, 262R-M, while the commonly reset control lines of the rowsincluding color sensitive pixels are designated with the referencenumeral 262, 262R-C. In the version of FIG. 5 f and FIG. 7 c, everyfourth row of pixels of image sensor array 182B includes clusters ofcolor sensitive pixels 257R, 257G, 257B. As shown in FIG. 7 c, withreset control lines 262, 262R-C of rows including color sensitive pixels250C electrically connected together, all rows of image sensor array182B including color sensitive pixels 250C may be driven into reset byapplication of a reset control signal on common reset control line 262,262R-C. Likewise, all rows of pixels including only monochrome pixels250M (the monochrome row of pixels) can be driven into reset by applyinga reset control signal on common monochrome pixel reset control line262, 262R-M. With further reference to the version of image sensor array182B shown in FIG. 7 c, monochrome pixels 250M of image sensor array182B may be driven into reset when pixels 250C are exposed for imagedata read out of color image data.

It is noted that with the configuration of FIG. 7 c, adjacent monochromepixels 250M-A adjacent to an end pixel, e.g., pixel 250C-E of a colorsensitive pixel cluster, e.g., cluster 257R are not driven into resetduring exposure periods of color sensitive pixels 250C. However,according to the invention in one example, image data corresponding onlyto a center pixel 250C-I of each color sensitive horizontally arrangedcluster (and not the end pixels 250C-E) may be selectively addressedduring read out of color image data. The presence of each lateral colorfilter element at the end pixels 250C-E, which are not addressed forimage data read out, reduces the effect of cross talk attributable tophotons entering image sensor array 182B at an angle through end pixels250C, 250C-E.

Another configuration for providing separately and independentlyresetting monochrome pixels 250M and color sensitive pixels 250C ofimage sensor array 182B is shown and described with reference to FIG. 7d. In the version of FIG. 7 d, image sensor array 182B includes aplurality of rows of pixels including all monochrome pixels 250Mfollowed by a plurality of rows of pixels that include color sensitivepixels 250C only. The monochrome rows of pixels 250M form a first subsetof pixels and the color sensitive pixels 250C form a second subset ofpixels. The reset control lines for resetting the first subset of pixelscan be made separate and independent of the reset control lines forcontrolling the second subset of pixels by electrically connecting thereset control lines of the first subset of pixels together and thenseparately electrically connecting together the reset control lines ofthe second subset of pixels. The common control lines of the firstsubset of monochrome pixels 250M in the version of FIG. 7 d aredesignated with reference numeral 262, 262R-M while the common controllines of the second subset of color sensitive pixels 250C in the versionof FIG. 7 d are designated with the reference numeral 262, 262R-C. Itwill be seen that the configuration of FIG. 7 d facilitating separateand independent control of monochrome pixels 250M and color sensitivepixels 250C can be utilized with the line art symbol optimized versionsof image sensor array 182B shown and described in FIGS. 5 g-5 i having“zones” of monochrome or alternatively color sensitive pixels 250C thatextend entire rows of image sensor array 182B.

Referring to FIG. 7 d, color sensitive pixels 250C may be driven toreset during exposure periods for monochrome pixels 250M by applicationof a common reset control signal on reset control line 262, 262R-Mduring exposure of color sensitive pixels 250C for read out of colorimage data. Similarly color sensitive pixels 250C may be driven intoreset by application of a reset control signal on common reset controlline 262, 262R-C during exposure periods of monochrome pixels 250M forread out of image data from monochrome pixels 250M.

Features respecting specific embodiments of an image sensor arrayaccording to the invention have been described in connection with theviews of FIGS. 2 a-4 c (image sensor array 182A), and the views of FIGS.5 a-7 d (image sensor array 182B). General features of an image sensorarray which may be incorporated into optical reader 100 (that is,features which can be incorporated in the image sensor array, whether ofthe embodiment labeled 182A, the embodiment labeled 182B, or anotherembodiment such as CMY image sensor array 182C, RGB image sensor array182D, monochrome linear image sensor array 182E, monochrome area imagesensor array 182F, monochrome and polarizer image sensor array 182G, ormonochrome color and polarizer image sensor array 182H) are nowdescribed.

Optical reader 100 can be programmed or otherwise be configured toselectively address a first plurality of pixels in an image sensor array182A, 182B, 182C, 182D, 182E, 182F, 182G, 182H independently ofselectively addressing a second plurality of pixels of the image sensorarray so that image data can be read out of the first plurality ofpixels independently of the second plurality of pixels. In one operatingmode optical reader 100 selectively addresses the first subset of pixelsand reads out image data from first subset of pixels independently ofthe second color sensitive subset of pixels. In another operating modeoptical reader 100 selectively addresses the second subset of pixels andreads out image data from the second subset of pixels independently ofthe first subset of pixels 250M. Where optical reader 100 selectivelyaddresses and selectively reads out only a subset of pixels of an imagesensor array, the resulting frame of image data read out of the imagesensor array may be referred to as a “windowed frame” of image data.When a windowed frame of image data is read out, the frame rate of theimage sensor array is normally increased relative to a normal frame rateof the image sensor array.

Image sensor array 182A, 182B, 182C, 182D, 182E, 182F, 182G, 182H can beconfigured to have a rolling shutter operating mode and a global shutteroperating mode. When a rolling shutter operating mode is entered, rowsof pixels of image sensor array are exposed sequentially. The term“rolling” shutter is used because when in a rolling shutter operatingmode an exposure period for a row of pixels generally begins prior to atime an exposure period for a previous row has ended.

When operated in a global electronic shutter operating mode, pixels fromseveral rows of an image sensor array are exposed simultaneously. Thatis, when operated in a global electronic shutter operating mode,transistor components (for example, transfer gates 304 and reset gates308 of the array as shown in the embodiments of FIGS. 3 a and 6 a)forming an electronic shutter an image sensor array are controlled in acoordinated manner so that a plurality of rows of pixels are exposedsimultaneously and have common exposure periods. In a global electronicshutter operating mode, electronic shutter components of the array arecontrolled so that the common exposure period for each of the pluralityof rows of pixels begins at a common exposure start time (via control ofreset gates 308) and ends at a common exposure stop time (via control oftransfer gates 304). As explained herein, each pixel of the array maystore a charge in an optically shielded storage region during the commonexposure period. For facilitating a global electronic shutter operatingmode, an exposure control timing pulse 354, 354′, 354″, 354″′ can beapplied to an image sensor array 182A, 182B, 182C, 182D, 182E, 182F,182G, 182H, as is described in further detail in connection with thetiming diagrams of FIGS. 15 a-15 e. An exposure control timing pulse354, 354′, 354″, 354″′ controls the timing for exposure of each row ofpixels of image sensor array 182A, 182B, 182C, 182D, 182E, 182F, 182G,182H being exposed. The exposure period for each row of pixels the imagesensor array being subject to image data read out begins at the leadingedge of exposure control timing pulse 354, 354′, 354″, 354″′ and ends atthe falling edge of exposure control timing pulse 354, 354′, 354″,354″′. For construction of an image sensor array having a globalelectronic shutter operating mode each pixel of the array is equippedwith additional circuit elements as is described herein.

Image sensor array 182A, 182B, 182C, 182D, 182E, 182F, 182G, 182H ofoptical reader 100 may be constructed to be operated in a rollingshutter mode of operation only; that is, in one specific embodiment animage sensor array of optical reader 100 can only be controlled toexpose pixels of the image sensor array on a rolling shutter basis andcannot be controlled so that pixels of image sensor array are exposed ona global electronic shutter basis. In another embodiment, an imagesensor array incorporated in optical reader 100 is constructed to beoperated in a global electronic shutter operational mode only and isincapable of operation in a rolling shutter mode.

Image sensor array 182A, 182B, 182C, 182D, 182E, 182F, 182G, 182H can beconstructed to be operated in either of a global electronic shutteroperation mode or a rolling shutter operational mode. Where an imagesensor array incorporated in optical reader 100 is constructed to beoperational in either of a rolling shutter operational mode or a globalshutter operational mode, the switching between rolling shutter andglobal shutter operational modes may be made in response to a receipt ofoperator instructions to change the shutter mode. The switching betweenrolling shutter and global shutter operational modes may also beautomatic and dynamic in response to the sensing of a predeterminedcriteria being satisfied. An image sensor array equipped optical reader100 having both rolling shutter and global shutter operational modes isdescribed in U.S. patent application Ser. No. 11/077,975, filed Mar. 11,2005 entitled, Bar Code Reading Device With Global Electronic ShutterControl, which is incorporated herein by reference. An image sensorarray constructed to be operated in either of a rolling shutter orglobal shutter operational mode is described in U.S. Pat. No. 6,552,323entitled, “Image Sensor With A Shared Output Signal Line” which isincorporated by reference.

Image sensor array 182A, 182B, 182C, 182D, 182E, 182F, 182G, 182H can beconstructed so that certain pixels of the image sensor array are capableof being exposed on either a rolling shutter basis or a global shutterbasis and certain other pixels of the image sensor array are capable ofbeing exposed only on a rolling shutter basis and are not capable ofbeing exposed on a global electronic shutter basis.

It has been described with reference specifically to image sensor array182A, and image sensor array 182B that it may be advantageous toincorporate into an image sensor array of optical reader 100 separatelycontrollable reset control lines 262, 262R-M and 262, 262R-C forresetting monochrome pixels separately and independently of colorsensitive pixels to thereby reduce pixel cross talk. It will beunderstood that it may be advantageous to incorporate separately andindependently controllable reset control lines into an image sensorarray 182A, 182B, 182C, 182D, 182E, 182F, 182G, 182H according to theinvention whenever image data is selectively read out of a first subsetof image data and it is desired to reduce cross talk from pixels of theimage sensor array external to the first subset of pixels. For example,in optical reader 100 incorporating a cyan-magenta-yellow (CMY) imagesensor array 182C as shown in FIG. 18 c, it may be advantageous toincorporate separate reset control lines for resetting magenta and cyanpixels separately from yellow pixels so that when yellow pixels areexposed for read out of decode frame yellow pixel image data fortransmission to decode circuit 1792, the remainder of the pixels of thearray, i.e., the cyan and magenta pixels can be set to reset toeliminate electron diffusion cross talk and to reduce photon penetrationcross talk. When hand held optical reader 100 incorporates a hybridmonochrome and polarizer image sensor array 182G as shown in FIG. 19 bor the monochrome color and polarizer image sensor array 182H as shownin FIGS. 20 a and 20 b, it may be beneficial to incorporate into imagesensor array 182 separately controllable reset control lines forcontrolling the reset of pixels external to the polarizing pixels sothat when the polarizing pixels are exposed for read out of image datafrom the polarizing pixels, the remaining pixels of the image sensorarray are set to reset to reduce cross talk from the pixels external tothe polarizing pixels.

While an image sensor array 182A, 182B, 182C, 182D, 182E, 182F, 182G,182H is conveniently provided by a CMOS image sensor array fabricatedutilizing complementary metal-oxide-silicone integrated circuitfabrication technologies, an image sensor array 182A, 182B, 182C, 182D,182E, 182F, 182G, 182H may also be a charge coupled device (CCD) imagesensor array, or a CID image sensor array or an image sensor array ofanother fabrication technology. In various embodiments of the inventiondescribed herein, it is advantageous to read out less than a full frameof image data, i.e., a read out of a “windowed frame” of image datawhich may also be referred to as an image region of interest (ROI). Anexample of a CCD image sensor array integrated circuit chip havingwindowing capability is the KODAK KAI-0340 image sensor array IC chipavailable from Eastman Kodak Corporation of Rochester, N.Y. The KAI-0340image sensor array IC chip has various operational modes that areselectable utilizing various input switch setups. For example, setting aSW1 switch to the HIGH position causes charge in outer verticalresisters of the image sensor array to be dumped before it reaches thehorizontal register, facilitating the selective read out of image datafrom center columns of the array only. Setting the SW2 switch of theKAI-0340 image sensor array chip changes diode transfer clock timingsuch that only charge from the center rows is transferred to verticalregisters, facilitating the selective read out of image data from centerrows of the image sensor array only. Accordingly, where image sensorarray 182B is configured according to the version shown in FIG. 5 hhaving center rows of monochrome pixels defining a monochrome pixel zone2500M and where the image sensor array is a CCD KAI-0340 image sensorarray, image data from the center rows can be read out by selecting apreconfigured operating mode of the image sensor array chip. Additional“windowed frame” patterns can be selectively read out of a CCD imagesensor array by varying the speed of a pixel clock timing control timingpulse that controls the speed with which a pixel is clocked. Invalid ornull data can be clocked out of a CCD pixel by speeding up a pixel clocksignal. Varying a pixel clock control signal between valid data yieldingrates and invalid data yielding rates during the reading out of imagedata from a CCD image sensor array yields a windowed frame of image datacomprising valid image data clocked out at normal speed and invalidimage data clocked out at high speed. Image data can also be selectivelyread out of a CCD image sensor array by selectively gating to outputcircuitry of the CCD image sensor array image data corresponding toselect pixels of the image sensor array. It will be seen that for anyapplication described herein wherein a windowed frame of image data isread by selective addressing of pixels from a CMOS image array, a CCDimage sensor array supporting windowing capability may be substitutedtherefore to provide selective read out functionality.

Additional aspects of the invention are described with reference to thephysical form views of FIGS. 8 a-8 c and the physical form views 9 a, 9b and 9 c. In the physical views of FIGS. 8 a-8 c, an imaging moduleonto which an image sensor chip may be incorporated is described. Withreference to FIGS. 9 a, 9 b and 9 c, hand held housings for supportingand encapsulating an imaging module including an image sensor chip aredescribed.

An optical reader 100 of the invention, as shown in the embodiment ofFIGS. 8 a-8 c, may include an imaging module such as imaging module1802A. Imaging module 1802A as shown in FIGS. 8 a-8 c incorporatescertain features of an IT4000 imaging module herein and additionalfeatures. IT4000 imaging modules are available from Hand Held Products,Inc. of Skaneateles Falls, N.Y. Imaging module 1802A includes firstcircuit board 1804 carrying light sources 160 a, 160 b, while secondcircuit board 1806 carries light sources 160 c, 160 d, 160 e, 160 f, 160g, 160 h, 160 i, 160 j, 160 k, 1601, 160 m, 160 n, 160 o, 160 p, 160 q,160 r, 160 s, and 160 t (hereinafter 160 c through 160 t). First circuitboard 1804 also carries image sensor array 182, which is integrated ontoimage sensor IC chip 1082. Image sensor IC chip 1082 and image sensorarray 182 in FIG. 8 a are generically labeled with the referencenumerals “1082” and “182” respectively in FIGS. 8 a-8 d to indicate thatany one of the specifically described image sensor IC chips 1082A,1082B, 1082C, 1082D, 1082E, 1082F, 1082G, 1082H described herein or anyone of the specifically described image sensor arrays 182A, 182B, 182C,182D, 182E, 182F, 182G, 182H described herein may be incorporated intoimaging module 1802A. The various image sensor IC chips and image sensorarrays can also be incorporated in another imaging module describedherein such as imaging module 1802B, 1802C, 1802D, and 1802E. Imagingmodule 1802C shown in FIG. 8 e is a laser aiming IT4300 imaging moduleof the type available from Hand Held Products, Inc. The laser aimingIT4300 imaging module includes a plurality of illumination LEDs e.g.,LED 160, and an aiming pattern generator comprising a laser diodeassembly 1872 in combination with a diffractive element 1873, whereinthe diffractive element of the imaging module diffracts laser light fromthe laser diode assembly to project a two-dimensional aiming patternonto a substrate, s. Imaging module 1802A also includes support assembly1810 including lens holder 1812, which holds lens barrel 1814 thatcarries imaging lens 212 that focuses images onto an active surface ofimage sensor array 182. Lens 212 may be e.g., a single lens (a lenssinglet), a lens doublet or a lens triplet. Light sources 160 a, 160 bare aiming illumination light sources whereas light sources 160 cthrough 160 t are illumination light sources. Referring to FIG. 8 d,illumination light sources 160 c through 160 t project a two-dimensionalillumination pattern 1830 over a substrate, s, that carries a decodableindicia such as a bar code symbol 1835 whereas aiming illumination lightsources 160 a, 160 b project an aiming pattern 1838. In the embodimentsshown and described in connection with FIGS. 8 a-8 c, light from aimingillumination light sources 160 a, 160 b is shaped by slit apertures 1840in combination with lenses 1842 which image slits 1840 onto substrate,s, to form aiming pattern 1838 which in the embodiment of FIGS. 8 a-8 cis a line pattern 1838. Illumination pattern 1830 substantiallycorresponds to a full frame field of view of image reader 100 designatedby box 1850. The present field of view of optical reader 100 is hereinreferred to as the “target” of optical reader 100. Aiming pattern 1838is in the form of a line that extends horizontally across a center offield of view of image reader 100. Illumination pattern 1830 may beprojected when all of illumination light sources 160 c through 160 t areoperated simultaneously. Illumination pattern 1830 may also be projectedwhen a subset of light sources 160 c through 160 t are simultaneouslyenergized. Illumination pattern 1830 may also be projected when only oneof light sources 160 c through 160 t is energized such as LED 160 s orLED 160 t. LEDs 160 s and 160 t of imaging module 1802 have a widerprojection angle than LEDs 160 c through 160 t. In an optical reader 100incorporating imaging module 1802, 1802A illumination assembly 104includes LEDs 160 a, 160 b, LEDs 160 c through 160 t and slit apertures1840 in combination with lenses 1842.

A reader imaging module may be incorporated into one of a hand heldhousing as shown in FIGS. 9 a, 9 b and 9 c. In the embodiment of FIG. 9a, hand held housing 101 is a gun style housing. In the embodiment ofFIG. 9 b, hand held housing 101 supporting imaging module 1802 is in theform factor of a portable data terminal (PDT). In the embodiment of FIG.9 c, hand held housing 101 supporting imaging module is in the formfactor of a mobile telephone, often referred to as a “cell phone.” Whenoptical reader 100 is a cell phone, optical reader 100 is configured tosend voice data over GSM/GPRS transceiver 571 to GSM/GPRS network 198(FIG. 10) and to receive over GSM/GPRS transceiver 571 voice data fromGSM/GPRS network 198. Further, where optical reader 100 is a cell phone,optical reader 100 may be configured so that an operator inputstelephone numbers via keyboard 508. The specific imaging module 1802Adescribed in connection with FIGS. 8 a-8 c may be incorporated in theoptical reader shown in FIG. 9 a or the optical reader 100 shown in FIG.9 b or the optical reader 100 shown in FIG. 9 c. However, in theembodiment shown in FIG. 9 a, housing 101 supports and encapsulatesimaging module 1802B an imaging module of construction similar toimaging module 1802A, except that only two light sources 160 areincorporated into the imaging module. Housing 101 of the reader of FIG.9 b supports imaging module 1802 which is generically labeled element1802 to indicate that any one of the specific imager modules describedherein, e.g., 1802, 1802A, 1802B, 1802D, 1802E may be incorporated intoan optical reader according to the invention.

Referring to further aspects of optical reader 100, optical reader 100may incorporate a graphical user interface (GUI) 3170 enabling selectionbetween various operating modes. With GUI 3170 an operator moves pointer3172 to a selected icon and clicks on the icon to configure opticalreader 100 in accordance with an operating mode associated with theselected icon. Reader 100 may include pointer mover 512 (otherwisetermed a navigation matrix) to facilitate movement of the pointer 3172.Buttons 512B of pointer mover 512 facilitate selection of an icon of aGUI interface that is supported by incorporating a multitaskingoperating system (OS) into reader 100 such as WINDOWS CE. GUI 3172 maybe developed using various open standard languages as HTML/Java orXML/Java.

In the embodiment of FIG. 9 b, GUI 3170 includes a plurality of virtualselection buttons 3152, 3154, 3156, 3158, 3162, 3164. Selection ofrolling shutter icon 3152 configures reader 100 so that during a nextexposure period image sensor array 182 is operated in a rolling shuttermode. Selection of global shutter icon 3154 configures optical reader100 so that during a next exposure period image sensor array 182 isoperated in a global electronic shutter mode.

Selection of decode icon 3162 drives optical reader 100 into an indiciadecode mode so that a next time a trigger signal is received, opticalreader 100 captures a frame of image data and attempts to decode a barcode symbol or other decodable indicia (e.g., an OCR character)represented therein and outputs a decoded out message to display 504, ora spaced apart device 150, as is described with reference to FIG. 10.Selection of image capture (which may otherwise be referred to a picturetaking) icon 3164 configures optical reader 100 so that next time atrigger signal is received, optical reader 100 captures image data andoutputs the image data to one or more of a display 504, a specifiedmemory address, or to a spaced apart device 150 without attempting todecode decodable indicia therein. Optical reader 100 may also beconstructed so that optical reader 100 can be configured in accordancewith a selected operating mode by sending to reader 100 a serial commandfrom a spaced apart device, or by the reading of a specially configuredprogramming bar code symbol.

Optical reader 100 is configured so that optical reader 100 receives atrigger signal when manual trigger 216 is manually depressed by anoperator. Optical reader 100 may also be configured so that a triggersignal is received by the sensing of an object in the proximity ofreader 100 or by the sending of a serial trigger command to reader 100from a spaced apart device, 150, as shown in FIG. 10.

A flow diagram illustrating operation of optical reader 100 in oneembodiment is described with reference to FIGS. 14 a, 14 b, and 14 c. Atstep 1100 an operator selects between an indicia decode mode and apicture taking mode. At step 1100 an operator may select icon 3162 (FIG.9 b) to drive optical reader 100 into an indicia decode mode, oralternatively icon 3164 to drive optical reader 100 into a digitalpicture taking mode of operation. These modes may also be selected bysending to reader 100 a serial command from a spaced apart device 150 orby the reading of a programming bar code symbol. If an indicia decodemode of operation is selected, optical reader 100 executes an indiciadecode process 1102. If a picture taking mode is selected, opticalreader 100 executes picture taking process 1400.

An example of an indicia decode process 1200 is described with referenceto FIG. 14 b. At step 1202 a trigger signal is received by one of themethods described (depressing trigger 216, object sensing, serialtrigger command) to commence a decode process. At step 1203, controlcircuit 552 of optical reader 100 captures a plurality of “parameterdetermination” or test frames of image data. The frames of image datacaptured at step 1203 are not subject to indicia decode processing, butrather, are processed for parameter determination (e.g., exposure, gain,illumination). Alternatively, parameter determining step 1203 may beavoided. For example, control circuit 552 may apply parametersdetermined from a previous image capture operation rather thandetermining parameters at step 1203. At step 1204 control circuit 552obtains a decode frame of image data details of which are explainedherein.

For the capturing of frames of image data (i.e., “test” frames and/orframes for use in decoding, picture taking or other processing orstorage) control circuit 552 (FIG. 1 a) may send an illumination controlsignal to illumination assembly 104 and various image capture initiationcontrol signals to control/timing circuit 1092 of image sensor chip 1082(labeled generically to refer to any of the image sensor chips describedherein).

Image capture initiation control signals are described in greater detailwith reference to FIGS. 15 a-15 e. For the capturing of image data,control circuit 552 may send to illumination assembly 104 anillumination control timing pulse 350 to energize at least one lightsource 160 such that illumination pattern 1830 is projected (as shown inFIG. 8 d). Control circuit 552 may also send to image sensor IC chip1082 an exposure control timing pulse 354 and a read out control timingpulse 368, and a reset control timing pulse 370 (that is, controlcircuit 552 sends appropriate signals to image sensor IC chip 1082 toinitiate exposure control timing pulse 354, read out control timingpulse 368 and reset control timing pulse 370).

In one embodiment as shown in FIG. 15 a, the exposure control timingpulse 354 begins after and finishes before the illumination controltiming pulse 350. The read out control timing pulse 368 begins at theconclusion of the illumination control timing pulse 350. In anotherembodiment as shown in FIG. 15 b, the illumination control timing pulse350′ begins after and finishes before the exposure control timing pulse354′. In this embodiment, the read out control timing pulse 368′ beginsat the conclusion of the exposure control timing pulse 354′. In furtherembodiments the exposure control timing pulse and the illuminationcontrol timing pulse overlap each other while occurring sequentially. Inone such embodiment as shown in FIG. 15 c, this sequential operation caninclude the illumination control timing pulse 350″ starting, theexposure control timing pulse 354″ starting, the illumination controltiming signal pulse 350″ ending, and then the exposure control timingpulse 354″ ending. In this embodiment, the read out control timing pulse368″ begins at the conclusion of the exposure control timing pulse 354″.In a further such embodiment as shown in FIG. 15 d, the sequentialoperation can include the exposure control timing pulse 354′″ starting,the illumination control timing pulse 350′″ starting, the exposurecontrol timing pulse 354″′ ending, and then the illumination controltiming signal pulse 350″′ ending. In this embodiment, the read outcontrol timing pulse 368″′ begins at the conclusion of the illuminationcontrol timing signal pulse 350″′. Each illumination control timingpulse 350, 350′, 350″, 350′″ described herein may comprise a pluralityof short duration individual pulses, sometimes referred to as a“strobed” pulse, as is indicated by FIG. 15 e.

When exposure control timing pulse 354 is received by an image sensor ICchip and optical reader 100 is configured in a global electronic shutteroperating mode, pixels from several rows of image sensor array 182A,182B, 182C, 182D, 182E, 182F, 182G, 182H are simultaneously exposed tolight for the duration of the pulse. That is, when optical reader 100 isconfigured in an global electronic shutter operating mode, each ofseveral rows of image sensor array 182A, 182B, 182C, 182D, 182E, 182F,182G, 182H that are subject to image data read out have common exposureperiods such that the exposure period for each row of pixels the imagesensor array subject to image data read out begins at a common exposurestart time and ends at a common exposure stop time. The exposure periodfor each row of pixels image sensor array 182A, 182B, 182C, 182D, 182E,182F, 182G, 182H subject to image data read out begins at the leadingedge of exposure control timing pulse 354, 354′, 354″, 354″′ and ends atthe falling edge of exposure control timing pulse 354, 354′, 354″,354′″.

When read out control timing pulse 368 is received by image sensor ICchip 1082B, image data is read out from the image sensor array. Imagesignals corresponding to pixels of the image sensor array are convertedinto digital form by analog-to-digital converter 1086 and transferredinto memory 560 by FPGA 580.

Optical reader 100 may be configured so that at step 1204 when readingout image data, optical reader 100 reads a “windowed frame” of imagedata. As indicated herein, a windowed frame of image data may be readout by selectively addressing pixels of a desired region of interest orwindow. A windowed frame of image data read out during frame capture atstep 1204 may comprise pixel values corresponding to all orsubstantially all monochrome pixels 250M of an image sensor array. Withfurther reference to the timing diagrams of FIGS. 15 a, 15 b, 15 c and15 d, a reset control timing pulse 370 for resetting pixels that are notselectively addressed may be coordinated with the exposure controltiming pulse 354 for controlling exposure of pixels that are selectivelyaddressed for reading out a windowed frame of image data. Thus, forreading out a monochrome frame of image data from a hybrid monochromeand color image sensor array, e.g., image sensor array 182A or imagesensor array 182B, reset control timing pulse 3709 is applied to resetcolor pixels of the image sensor array 182 while exposure control timingpulse 354 is applied to enable exposure of monochrome pixels of theimage sensor array. To facilitate resetting of color pixels of an imagesensor array independent of resetting of monochrome pixels, an imagesensor array may be configured to include a reset control line gridspecifically adapted to enable resetting of color pixels. Applying resetcontrol pulse 370 to drive color pixels into reset while monochromepixels are being exposed to light can be expected to eliminate electrondiffusion cross talk and to reduce cross talk resulting from light raysangularly entering the color pixels during exposure.

When frames are obtained at step 1204, they are obtained in a formsuitable to facilitate indicia decoding such as bar code symbol decodingor OCR decoding. With the windowed frame of image data read out at step1204 from a hybrid monochrome and color image sensor array 182A, 182Bincluding only image data corresponding to monochrome pixels and noimage data corresponding to color sensitive pixels 250C, control circuit552 at step 1204 may store gray scale values into RAM 560, each pixelvalue representing an intensity of light at a particular monochromepixel of image sensor array 182A, 182B. The frame of image data obtainedat step 1204 may include e.g., 8 bit gray scale pixel values, 10 bitgray scale pixel values or 12 bit gray scale pixel values. Sincenumerous legacy bar code decoding and OCR decoding schemes are designedto operate on monochrome gray scale image data or binarized image dataderived from gray scale image data, the selective addressing ofmonochrome pixels 250M in the capturing of a monochrome image frameyields a frame that is well suited for being subjected to indiciadecoding processing. Of course, in certain applications, control circuit552 at step 1204 may obtain a decode frame of image data including colorimage data. For example, where decode circuit 1702 is configured todecode color encoded bar code symbols, it is advantageous for controlcircuit 552 to obtain a decode frame of image data including color imagedata at step 1204.

In the execution of step 1204, control circuit 552 may carry out aplurality of alternative processes in obtaining a decode frame of imagedata. Referring to the flow diagram of FIG. 14 d, optical reader 100 atstep 1204 may simply capture a single windowed frame of image data whichhas been described herein above. As indicated by process step 1205 ofFIG. 14 d, control circuit 552 may carry out process step 1204 byselectively addressing monochrome pixels 250M of a hybrid monochrome andcolor image sensor array such as image sensor array 182A or image sensorarray 182B and reading out image data from monochrome pixels 250M only;i.e., a windowed frame of image data comprising image data frommonochrome pixels 250M only.

Referring to the flow diagram of FIG. 14 e, the obtaining a decode framestep 1204 may be carried out in the alternative by the execution ofsteps 1206 and 1207. At step 1206 optical reader 100 may generate aframe of image data that includes image data corresponding to monochromepixels 250M and color sensitive pixels 250C and at step 1207 imagesensor array 182A, 182B may convert pixel values of the frame generatedat step 1206 into gray scale values. The frame generated at step 1206may be generated by exposing color and monochrome pixels of image sensorarray 182A, 182B during a single exposure period, and reading out imagedata from both color and monochrome pixels 250M, 250C of image sensorarray 182A, 182B during a single pixel read out period. Alternatively,at step 1206 control circuit 552 of optical reader 100 may combine imagedata from two different frames such as two successive frames, wherein afirst of the captured frame is a windowed frame of image data includingimage data from color sensitive pixels 250C only and a second of theframes is a frame of image data including image data read out frommonochrome pixels 250M only.

Referring to the flow diagram of FIG. 14 f, optical reader 100 may alsoobtain a decode frame at step 1204 by carrying out steps 1208 and step1209. At step 1208 optical reader 100 may capture a windowed frame ofimage data including image data corresponding to monochrome pixels 250Monly and at step 1209 control circuit 552 may interpolate pixel valuescorresponding to color pixel positions of image sensor array 182A, 182Butilizing the monochrome pixel values from the windowed monochrome framecaptured at step 1208. For example, control circuit 552 may capture agray scale pixel value frame 5202 as illustrated in FIG. 16 a thatincludes a gray scale pixel value for each monochrome pixel position ofthe image sensor array 182A, 182B. Optical reader 100 may interpolate amonochrome pixel value for any “missing pixel” color pixel position ofthe frame 5202. Referring to frame 5202, frame 5202 is a gray scaleframe of image data captured by selecting reading out image data from animage sensor array 182B constructed in accordance with FIGS. 4 a-7 b(Period=2). Pixel positions P₁₁, P₃₁, P₅₁, P₁₂, P₂₂, P₃₂, P₄₂, P₅₂, P₁₃,P₃₃, P₆₃ . . . are pixel positions corresponding to monochrome pixels250M of image sensor array 182 for which individual frame image data hasbeen read out. Pixel positions P₂₁, P₄₁, P₂₃, P₄₃, . . . are missingpixel positions corresponding to color sensitive pixels, 250C of imagesensor array 182B. Referring to the frame of image data represented atFIG. 16 a an optical reader 100 may calculate a gray scale pixel valuefor color pixel positions, e.g., position P₂₃, by averaging the grayscale values for each pixel position that is adjacent to pixel positionP₂₃ and each pixel position that is corner adjacent to color pixelposition P₂₃. For example, referring to the frame represented in FIG. 16a, a gray scale value for color pixel position P₂₃ can be interpolatedby averaging pixel values of pixel positions P₁₂, P₂₂, P₃₂, P₁₃, P₃₃,P₁₄, P₂₄, P₃₄. A pixel value for “missing pixel” position P₂₃ can alsobe interpolated utilizing more than 8 neighboring pixel positions. Also,corner adjacent pixels may be weighted less than side, top or bottomadjacent pixels during averaging. In one simple averaging method, onlyfour surrounding pixels are averaged; namely, the top and bottomadjacent pixels and the two side adjacent pixels adjacent to the pixelposition for which a gray scale value is being interpolated. In a stillfurther interpolation method, only two pixels are used for averaging;namely either the two side adjacent pixels adjacent to the pixelposition being interpolated or the top and bottom adjacent pixels. Atwo-dimensional image representation of a linear bar code symbol can beexpected to have several consecutive pixel positions along a column withsimilar gray scale values, if the representation of the symbol isoriented with 0° or 180° angle of rotation (i.e., the symbol is rightside up or upside down). If the symbol representation has a 90° or 280°angle of rotation, several consecutive pixel positions along rows ofpixel positions can be expected to have similar valued gray scalevalues. Accordingly, it can be seen that interpolating pixel values ofadjacent pixel position values running in the direction of bars in asymbol representation yields truer edge information than utilizing allsurrounding pixel positions for interpolation.

In one method of the invention, the correlation between a pair ofhorizontally oriented scan lines is calculated along with a correlationbetween a pair of vertically oriented scan lines. The two correlationmeasurements are then compared. If row scan lines are more closelycorrelated, column adjacent pixels are selected for interpolation. Ifcolumn scan lines are more closely correlated, row adjacent pixels areselected for interpolation. An exemplary set of code for calculating afirst derivative correlation for a pair of scan lines (horizontal orvertical) is presented by Table 1.

TABLE 1 Exemplary Code For Performing First Derivative CorrelationCalculation %OneDcorelate: correlates two 1D 1st derivative signals toreport the %correlation %input a,b: 1D array %output c: 1st derivativecorrelation function c=OneDcorrelate(a,b) % “diff” is the firstderivative calculation. %for an input array a=[a_(i)]_(i=1) ^(n) thendiff(a) =[a_(i) − a_(i+1)]_(i=1) ^(n−1) da=diff(double(a));db=diff(double(b)); n=length(da); c=0; for i=1:n c=c+da(i)*db(i); end[End Table 1]

A set of code for interpolating missing color pixel position values byone of three methods (simple averaging, first derivative correlation,and simple correlations) wherein “M-set” refers to the monochrome set ofpixels is presented in Table 2.

TABLE 2 Exemplary Code For Interpolating Missing Pixels Corresponding ToColor Pixel Positions % MsetInterpolation: interpolates missing M-setpixels %input I_Mset: M-set image %input method: 1:first derivativecorrelation; 2: simple correlation; 3: %simple averaging %input p:sample period %output Im: interpolated monochrome image functionIm=MsetInterpolation(I_Mset,method,p) Isz=size(I_Mset); %M-set topology% {circumflex over ( )} % MMMMMMMMM % MxMxMxMxM % MMMMMMMMM % MxMxMxMxM%(MMMMMMMMM) % v Im=double(I_Mset); m=Isz(1); n=Isz(2); %correlatedaveraging for i=p:p:m for j=p:p:n if i+1 <=m & j+1 <=n if method == 2%simple correlation if abs(Im(i−1,j)−Im(i+1,j)) <abs(Im(i,j−1)−Im(i,j+1)) Im(i,j)=(Im(i−1,j)+Im(i+1,j))/2; elseIm(i,j)=(Im(i,j−1)+Im(i,j+1))/2; end else if method == 1 %firstderivative correlation ifOneDcorrelate(Im(i−1,j−1:j+1),Im(i+1,j−1:j+1)) >OneDcorrelate(Im(i−1:i+1,j−1),Im(i−1:i+1,j+1))Im(i,j)=(Im(i−1,j)+Im(i+1,j))/2; else Im(i,j)=(Im(i,j−1)+Im(i,j+1))/2;end else %method==3 %simple averagingIm(i,j)=(Im(i−1,j)+Im(i+1,j)+Im(i,j−1)+Im(i,j+1))/4; end else if i+1 <=m& j+1 > n Im(i,j)=(Im(i−1,j)+Im(i+1,j))/2; else if i+1 > m & j+1 <=nIm(i,j)=(Im(i,j−1)+Im(i,j+1))/2; else if i+1 > m & j+1 > nIm(i,j)=(Im(i−1,j)+Im(i,j−1))/2; end end end Im=uint8(Im); [End Table 2]

At step 1210, optical reader 100 transfers the frame of image dataobtained at step 1204, to an indicia decode circuit 1702 which may be abar code symbol decoding circuit or autodiscrimination circuit 1704including an indicia decode circuit 1702. In one embodiment, decodecircuit 1702 decodes 1D and 2D bar code symbols and OCR characters.Autodiscrimination circuit 1704 may decode 1D and 2D bar code symbolsand OCR characters (decodable indicia) and automatically discriminatebetween decodable indicia and handwritten characters. In the event thatautodiscrimination circuit 1704 recognizes the presence of handwrittencharacter information, autodiscrimination circuit 1704 automaticallyoutputs to display 504 and/or a spaced apart device 150 image datarepresenting the handwritten character image data. Further details ofindicia decode circuit 1702 and autodiscrimination circuit 1704 aredescribed in copending U.S. patent application Ser. No. 11/077,975,filed Mar. 11, 2005, incorporated by reference and U.S. application Ser.No. 10/958,779, filed Oct. 5, 2004, also incorporated herein byreference.

In general, indicia decoding accuracy is expected to increase with anincrease in the percentage of monochrome pixels of image sensor array182A, 182B. With image sensor array 182B having a period of P=2, everyother row of pixels of image sensor array 182B are all monochromepixels. Thus, horizontal scan lines can be launched through horizontalrows of pixel values of a frame of image data obtained utilizing a P=2image sensor array 182B during attempts to decode a linear bar codesymbol without substantial reduction in performance relative to a frameobtained utilizing an all monochrome pixel image sensor array. Fordecoding linear bar code symbols, control circuit 552 may read imagedata along scan lines, such as scan lines defined by a horizontal row ofpixel positions to determine the relative widths of bars and spaces of asymbol and then decode the symbol through table lookup to determine aset of decoded character data corresponding to the bar spaceinformation.

At step 1212 control circuit 552 receives a decoded output message fromdecode circuit 1702 or autodiscrimination circuit 1704. The messagereceived by control circuit 552 at step 1212 may be e.g., a decoded barcode message or a set of decoded OCR characters. At step 1214 opticalreader 100 outputs a decoded out message. At step 1214 control circuit552 may send decoded out bar code data and/or decoded OCR data todisplay 504 or to a spaced apart device 150 or to a data storage memorylocation of reader 100, or system 145 as described in FIG. 10.

Examples of spaced apart devices 150 which may be in communication withoptical reader 100 are shown and described in connection with FIG. 10.Optical reader 100 may be part of a system 145 and may be included in alocal area network (LAN) 170 which comprises, in addition to reader 100,such spaced apart devices as other portable readers 100′, 100″, networkaccess point 174, personal computers 172 and central server 176 that arespaced apart from hand held housing 101 of reader 100, all of which areconnected together via backbone 177. Server 176 in turn is incommunication with a variety of additional spaced apart devices 150 thatare spaced apart from hand held housing 101 of reader 100 and whichthrough server 176 are in communication with optical reader 100. Server176 may be connected via gateways 179, 180 and network 181 to a firstdistant remote local area network 185 located miles to thousands ofmiles away from local area network 170 and a second distant local areanetwork 2170 also located miles to thousands of miles away from localarea network. Network 170 may be located at a supplier's warehouse.Network 2170 may be located at a delivery destination; and network 185may be located at a data processing/data archiving facility. Network 185can be configured to assemble, store and maintain in server 184 variousweb pages accessible with use of optical reader 100, that summarize datathat has been collected by various optical readers 100, 100′, 100″,100R. Server 176 may alternatively or redundantly be connected to remotenetwork 185 via private communication line 190. IP network 181 may bethe Internet or a virtual private network (VPN). Remote LAN 185 mayinclude a personal computer 186 and a remote server 184 connected viabackbone 191. Remote LAN 185 may also include a wireless communicationaccess point 193. Remote LAN 185 may also include a personal dataassistant (PDA) 189. Remote LAN 2170 may include a server 2176,connected to IP network 181 via gateway 2179, backbone 2177, accesspoint 2174, PC 2172, and optical reader 100, 100R. System 145 may beconfigured so that a display equipped device, e.g., device 100′, 172,186, 189 automatically displays data, such as decoded out bar codemessage data of a visual display color image frame of image data,received from optical reader 100 on its associated display 1504 whenreceiving that data.

All of the steps of process 1200 are carried out automatically inresponse to the receipt of a trigger signal. The steps of process 1200continue automatically until a stop condition is satisfied. A stopcondition may be e.g., the receipt of a trigger stop signal as may begenerated by release of trigger 216 or the successful decoding of apredetermined number of bar code symbols. As indicated by return line1211 of FIG. 14 b, control circuit 552 may repeatedly attempt to obtainimage data and attempt to decode decodable indicia therein until a stopcondition is satisfied.

Interpolating monochrome pixel values for “missing pixels” pixelpositions is particularly advantageous where a hybrid monochrome andcolor image sensor array according to the invention includes a highnumber of color sensitive pixels distributed at spaced apart pixelpositions throughout image sensor array. In other instances as alludedto previously in connection with the flow diagram of FIG. 14 b, controlcircuit 552 may obtain a decode frame of image data at step 1204 that issuitable for transferring to decode circuit 1702 by simply reading froman image sensor array image data from monochrome pixels 250M withoutinterpolation of any pixel values at “missing pixel” pixel positionswhere an image sensor array of reader 100 is constructed in accordancewith image sensor array 182A, and monochrome pixels 250M form a completecheckerboard pattern, (there are no “missing” monochrome pixelspositions in the M×N matrix of monochrome pixels within image sensorarray 182A). Accordingly, where optical reader 100 includes image sensor182A, the decode frame of image data at step 1204 is advantageouslyobtained by reading out from image sensor array 182A image data frommonochrome pixels 250M without interpolation of any further monochromepixel values.

It may also be useful to obtain a decode frame of image data at step1204 without interpolation of monochrome pixel values where opticalreader 100 includes a linear bar code symbol optimized image sensorarray 182B of one of the versions described in connection with FIGS. 5g-5 j. In the versions of image sensor array 182B shown and described inconnection with FIGS. 5 g-5 j, image sensor array 182B includes at leastone “zone” of monochrome pixels 2500M and at least one “zone” of colorsensitive pixels 2500C. Monochrome zone of pixels 2500M of a linearsymbol decoding optimized version of image sensor array 182B generallycomprises an elongated line of monochrome of pixels 250M having a minordimension of one to a limited number of rows of pixels (which may bediagonal rows). Where optical reader 100 includes a linear symboloptimized version of image sensor array 182B a reduced area decode frameof image data at step 1204 without any “missing pixel” pixel positionscan be obtained by selectively addressing pixels of monochrome zone2500M and selectively reading out of image data from the pixels ofmonochrome zone 2500M without reading any image data from the pixels ofcolor sensitive zone of pixels 2500C. More specifically, where opticalreader 100 comprises a linear symbol optimized version of image sensorarray 182B, control circuit 552 at step 1204 may obtain a reduced areamonochrome frame of image data at step 1204 (FIG. 14 b) and thentransfer the reduced area monochrome frame of image data to decodecircuit 1702 at step 1210. A reduced area frame of image data is furtherexplained with reference to FIG. 11 illustrating an application where anoptical reader 100 is utilized to collect decoded bar code data andimage data from a parcel 1260 carrying various bar code symbols, e.g.,linear bar code symbol 1266 and two-dimensional bar code symbol 1270. Afull area frame of image data that may be obtained by optical reader 100represents the scene area indicated by rectangular border 1276 of FIG.11. Where image data from all pixels of image sensor array 182B are readout, a full area frame of image data is obtained. When optical reader100 obtains a reduced area frame of image data, a reduced area frame ofimage data representing the reduced scene area as indicated by dashed-inborder 1278 may be obtained. In the example of FIG. 11, optical reader100 may incorporate one of the linear symbol optimized image sensorarrays 182B as shown in FIGS. 5 g and 5 h. A reduced area frame of imagedata representing the reduced scene area 1278 may be obtained by readingout image data from monochrome zone of thin center line monochrome zone2500M of image sensor array 182B according to one of the versions ofFIGS. 5 g and 5 h. It is seen with reference to FIG. 11 that whenoptical reader 100 obtains a reduced area frame of image data at step1204 representing the reduced scene area 1278, the reduced area frame ofimage data, though reduced, can be of sufficient size to include arepresentation of linear bar code symbol 1266 carried by parcel 1260.Imaging module 1802 such as module 1802A (FIG. 8 a) of reader 100 can beconstructed so that aiming pattern 1838 (FIG. 8 d) is projected ontoscene area 1278 at expected reading angles, while aiming light sources160 a, 160 b, and remaining light sources 160 c-160 t are energizedsimultaneously during the time that pixels of zone 2500M are exposed forread out of image data from zone 2500M. Simultaneously projecting aimingpattern 1838 and illumination pattern 1830 onto scene area 1278 improvesthe signal strength of image data corresponding to pixels of zone 2500M.After receiving the thin line reduced area frame of image data at step1210, decode circuit 1702 may process the thin line reduced area decodeframe of image data to decode linear bar code symbol 1266 by calculatingthe bar space widths of the bars and spaces of linear bar code symbol1266 and then determining the characters of the symbol through tablelookup. In a further aspect, optical reader 100 may be configured sothat aiming pattern 1838 (FIG. 8 d) is projected horizontally at acenter of a field of view 1276 of optical reader 100 to coincide witharea 1278 represented by the reduced area image obtained at step 1204 toaid an operator in obtaining an image that includes a representation oflinear bar code symbol 1266. The frame rate of optical reader 100 whenobtaining the reduced area decode frame of image data at step 1204 maybe significantly reduced relative to the frame rate of optical reader100 when obtaining a full frame of image data. Accordingly, a method ofthe invention where optical reader 100 at step 1204 obtains a reducedarea frame of image data which is transferred to decode circuit 1702 isoptimized for fast (“snappy”) decoding. As has been described herein,color sensitive pixels 250C may be set to reset while monochrome pixels250M are exposed for selective read out of image data from monochromepixels 250M.

With further reference to the application view of FIG. 11, it is seenthat the reduced area frame of image data representing scene area 1278may not include a complete representation of linear bar code symbol 1266and it is further seen that parcel 1260 may include or carry additionalbar code symbols such as two-dimensional bar code symbol 1270 that ispart of postal area 1268 of parcel 1260. According to the invention inanother aspect, optical reader 100 can be configured so that whereindicia decode circuit 1702 cannot successfully decode a bar code symbolvia processing of a reduced area frame of image data or where controlcircuit 552 is programmed to search and decode multiple bar codesymbols, control circuit 552 executes return line (FIG. 14 b) tore-execute the obtaining of a decode frame of image data at step 1204.However, when control circuit 552 executes step 1204 the second time,control circuit 552 captures a frame of image data that represents ascene area that is larger than the scene area represented by the frameobtained during the first execution of step 1204. The decode frame ofimage data obtained by a second execution of step 1204 may be a fullarea image data frame representing the full field of view of opticalreader 100 indicated by dashed-in border 1276 of FIG. 11. Where colorzones 2500C of image sensor array 182B are distributed in a Bayerpattern, control circuit 552 during the second execution of obtain step1204 may selectively read out image data from the green pixels of colorsensitive zones of image sensor array of 2500C and interpolate greenpixels values at non-green pixel positions utilizing the green pixelvalues so that the decode frame of image data obtained at step 1204includes all green pixel values. Further, the missing pixel positionscorresponding to monochrome zone 2500M can be filled in utilizing theimage data obtained during the previous execution of step 1204 as scaledbased on a relationship between the color scale values of pixelscorresponding to zone 2500M and the color scale values of pixelssurrounding zone 2500M. At step 1210, larger area green image data istransferred to indicia decode circuit 1702. Indicia decode circuit 1702may attempt to decode linear bar code symbol 1266 and all other bar codesymbols such as two-dimensional bar code symbol 1270 that may berepresented in the image obtained during the second execution of step1204. With reference to the application view of FIG. 11, optical reader100 incorporating a linear symbol decode optimized to image sensor array182B may attempt to decode linear symbol 1266 utilizing small area imagerepresenting area 1278 and then subsequently attempt to decode atwo-dimensional bar code symbol, e.g., symbol 1270, utilizing a largerarea frame of image data representing scene area 1276. It will be seenthat the method described where control circuit 552 obtains a reducedarea frame of image data, attempts to decode, then subsequently obtainsa larger frame of image data and attempts to decode utilizing the largerimage may be practiced utilizing an “all monochrome” image sensor array182F as shown and described in connection with FIG. 17 b. Where opticalreader 100 incorporates an all monochrome image sensor array 182F asshown in FIG. 17 b, it is particularly useful to set monochrome pixels250M to reset that are not being selected for read out of a reduced areadecode frame of image data at step 1204 during exposure periods forselected monochrome pixels 250M that are being selectively addressed forimage data read out.

Monochrome pixels 250M transmit more light than color sensitive pixels250C. Therefore, resetting monochrome pixels 250M that are notselectively addressed and which are adjacent to a region of interestduring an exposure period can be expected to have a greater benefit interms of improving the overall signal to noise ratio of reader 100 thanresetting color sensitive pixels 250C that are adjacent to a region ofinterest during an exposure period.

With still further reference to the application view of FIG. 11, it maybe advantageous to utilize optical reader 100 to obtain a visual displaycolor frame of image data representing parcel 1260. For example, parcel1260 may include a damaged area 1272. Obtaining a visual display colorframe of image data corresponding to parcel 1260 creates a recorddocumenting parcel damage. Referring to the application view of FIG. 12a, different optical readers 100 and 100R at different locations A and Blocated miles apart along a delivery route may be utilized to documentphysical transformations of parcel 1260 when parcel 1260 is carriedalong a delivery route. Optical reader 100 at location A including LAN170 (FIG. 10) may be operated to obtain a visual display color frame ofimage data of parcel 1260 when parcel 1260 is located at location A.Further, the color frame may automatically be transferred to remoteserver 184 (FIG. 10) having a database 187 of color frames of image datathat are indexed by a parcel identifier decoded in a parcel bar codesymbol 1266 which identifier is also transmitted to remote server 184automatically when optical reader 100 reads bar code symbol 1266. Atlocation B remote optical reader 100, 100R (FIG. 10) may be utilized toagain decode bar code symbol 1266 and capture visual display color frameof image data representing parcel 1266 and automatically transfer theparcel identifier corresponding to bar code 1266 and the color frame ofimage data to remote server 184. With reference to the application viewof FIG. 12 a the color frame of image data transmitted to remote server184 from location B will include a representation of damaged area 1272that is not included in the color frame of image data transmitted toremote server 184 from location A. Accordingly, a person (for example,at PC 172 viewing web pages of server 184) reviewing the parcelidentifier indexed color frame data of database 187 can determine thatthe damage to parcel 1260 occurred during the time that the parcel wasdelivered from location A to location B. Referring to FIG. 12 b, opticalreader 100 can also be utilized to take color pictures of a deliveryvehicle 1282 that carried parcel 1260 from location A to location B. Inthe example of FIG. 12 b, a picture being taken by optical reader 100has the field of view indicated by rectangle 1286. The field of viewencompasses parcel 1260, and delivery vehicle 1282, including a licenseplate 1284. Trigger 216 can be actuated a first time to decode bar codesymbols 1266, 1267 and then an additional time or times to have apicture of parcel 1260 and/or vehicle 1272 including a picture oflicense plate 1284. The decoded bar code data and multiple color framesof image data may be associated with one another into a singletransaction data set, and then via a packet based transmission scheme,the transaction data set may be sent to remote server 184, which mayorganize the data into viewable web pages viewable at PC 172. Opticalreader 100, which may be incorporated in hand held housing 101, can beconfigured so that all of the data of the transaction data set is sentto remote server 184 in response to a single command input to opticalreader 100 via a user interface of reader 100 (e.g., 3170). Furtheraspects of optical reader 100 operating in a picture taking mode ofoperation are described with reference to the flow diagrams of FIGS. 14c, 14 g and 14 h.

Referring again to the flow diagram of FIG. 14 a, a digital picturetaking process 1400 is executed when optical reader 100 is configured tooperate in a picture taking mode of operation. At step 1100, a picturetaking mode of operation may be selected, e.g., by clicking on “imagecapture” icon 3164 (FIG. 9 b) and at step 1104 picture taking process1400 is executed.

Referring to the steps of picture taking process 1400, optical reader100 at step 1402 receives a trigger signal as may be generated e.g., bydepression of a manual trigger an object in the proximity of reader 100being sensed or receipt of a serial command. At step 1403 controlcircuit 552 captures a plurality of “test” or parameter determinationframes of image data. The frames of image data captured at step 1403 arenot output for visual display; but rather are processed in order todetermine operational parameters (exposure setting, gain illumination).Alternatively, step 1404 can be avoided and control circuit 552 caninstead load operational parameters that were derived during a pastimage capture operation. At step 1404 control circuit 552 obtains a“visual display” image frame of image data. A visual display color frameof image data is one that is generated for visual display on a displayand may include three color scale values for each pixel position of theplurality of pixel positions of the frame. A visual display frame afterbeing obtained is sent to a display for visual display of an image or toa memory location for future display. In the embodiment of FIG. 14 c,the image data obtained at step 1404 is not transferred to decodecircuit 1702.

An image captured as part of obtaining at step 1404 may be one that iscaptured in accordance with the timing diagrams of FIGS. 15 a-15 e. Inan alternative embodiment, the control signals input into image sensorIC chip 1082 for the capture of a frame of image data may not includeillumination control timing pulse e.g., pulse 350. In many applicationsan object subject to image capture by optical reader 100 during apicture taking mode will be a long range image (an object will besubject to image capture is one that is one to several feet from imagereader). Light from light sources 160 may have little affect on an imagecaptured that corresponds to a long range object; thus, optical reader100, in one embodiment may not send an illumination control timing pulseat step 1404.

However, depending on the application, it may be desirable to increasethe illumination intensity of optical reader 100 during capture of colorimage data relative to the intensity during capture of monochrome imagedata to compensate for the signal reduction affect of color filterelements 260R, 260G, 260B, 260M, 260C. In a further aspect, opticalreader 100 can have a plurality of operator selectable configurationsettings. Optical reader 100 can be configured so that activation ofbutton 3150 toggles through a sequence of options one of which may beselected by actuation of a key of keyboard 508. As shown by Table 3,where e=exposure, g=gain, and i=illumination intensity, a selection of aconfiguration setting can result in a differentiation between theimaging parameters of reader 100 during read out of monochrome imagedata at step 1204 and the imaging parameters of reader 100 during readout of color image data at step 1404. Configuration setting 1 is a baseline setting wherein there is no differentiation between monochrome readout and color image data read out imaging parameters. Configurationsetting 2 has been described above. With configuration setting 2, thereis no illumination during read out of color image data at step 1404.Configuration setting 3 has also been described above. Withconfiguration setting 3, illumination intensity is increased for readout of color image data. With configuration setting 4, illuminationintensity for read out of monochrome image data can be increased. Forexample, as described herein, illumination pattern 1830 and aimingpattern 1838 can be projected simultaneously during read out ofmonochrome image data corresponding to a monochrome zone 2500M ofpixels. With configuration setting 5, exposure time is boosted for readout of color image data and with configuration setting 6, gain isboosted for read out of color image data. Configuration setting 3 ishighly useful where optical reader 100 includes a long distance flashillumination light source 160, 160× or where optical reader 100 is usedfor picture taking at close range.

TABLE 3 Imaging Parameters When Reading Out Imaging Parameters WhenMonochrome Image Reading Out Color Image Data At Decode Data At VisualConfiguration Frame Obtain Step 1204 Display Obtain Step 1404 1 e = e₀ e= e₀ g = g₀ g = g₀ i = i₀ i = i₀ 2 e = e₀ e = e₀ g = g₀ g = g₀ i = i₀ i= 0 (Illumination Off) 3 e = e₀ e = e₀ g = g₀ g = g₀ i = i₀ i = i₁, i₁ >i₀ 4 e = e₀ e = e₀ g = g₀ g = g₀ i = i₂, i₂ > i₀ i = i₀ 5 e = e₀ e = e₁,e₁ > e₀ g = g₀ g = g₀ i = i₀ i = i₀ 6 e = e₀ e = e₀ g = g₀ g = g₁, g₁ >g₀ i = i₀ i = i₀

In executing the obtaining visual display color frame of image data step1404, optical reader 100 may carry out a variety of alternativeprocesses. With reference to the flow diagram of FIG. 14 g, a process isdescribed wherein optical reader 100 may obtain a visual display colorframe of image data utilizing image data read out from color sensitivepixels 250C only. With reference to the flow diagram of FIG. 14 h, aprocess is described wherein control circuit 552 obtains a visualdisplay color frame of image data utilizing image data derived byreading out of image data from both monochrome pixels and colorsensitive pixels of image sensor array 182.

Referring to the flow diagram of FIG. 14 g, control circuit 552 at step1405 captures a windowed frame of image data by selectively addressingcolor pixels 250C and by selectively reading out image data from colorpixels 250C of image sensor array 182A, 182B. As explained previouslyherein, image sensor array 182A, 182B may include a separate resetcontrol grid for resetting monochrome pixels 250M independent of colorsensitive pixels 250C. At step 1405 while color sensitive pixels areexposed for read out of image data, monochrome pixels 250M may be resetwith use of a reset control timing pulse 370, 370′, 370″, 370″′ (FIGS.15 a-15 d). Coordinating a reset control timing pulse 370, 370′, 370″,370″′ for resetting monochrome pixels with an exposure control timingpulse 354, 354′, 354″, 354″′ for controlling exposure of color sensitivepixels 250, 250C reduces cross talk resulting from light rays enteringmonochrome pixels 250M, i.e., by eliminating electron diffusion crosstalk and by reducing cross talk attributable to light rays angularlypenetrating through monochrome 250M.

At step 1406, optical reader 100 automatically transfers the colorfilter array image data frame captured at step 1405 to a demosaicingcircuit 1706 (FIG. 1 e). Taking as an input a color filter array imagedata frame, demosaicing circuit 1706 outputs a visual display colorframe of image data. Where display 504 is a color display configured toreceive red, green and blue (RGB) signals for each pixel of display 504,demosaicing circuit 1706 at step 1406 may generate RGB color scalevalues for each pixel of display 504 so that a frame output bydemosaicing circuit 1706 is compatible with display 504. The color scalevalues may comprise e.g., 8 bits, 10 bits, or 12 bits of data. At step1407, optical reader 100 receives a visual display color frame of imagedata from demosaicing circuit 1706.

A particular example of optical reader 100 executing step 1404 isdescribed with reference to FIG. 16 b. At step 1406 where optical reader100 includes a hybrid monochrome color image sensor array 182A, 182Bincluding a Bayer pattern color filter array as shown in FIG. 2 c andFIG. 5 a, optical reader 100 may read out an RGB Bayer pattern frame ofimage data as shown in FIG. 16 b. Where a reader image sensor array isprovided by image sensor array 182B including a 1280×1024, array ofpixels, with a 320×256 subset array (P=4) of color sensitive pixels 250,250C (P=4) dispersed in array 182B optical reader 100 at step 1405captures a 320×256 Bayer pattern of pixels. Demosaicing circuit 170processes the Bayer pattern frame 1502 as shown in FIG. 16 b to output avisual display color frame of image data including a 320×256 colorimage, wherein each pixel of the frame includes a red color scale value,a green color scale value, and a blue color scale value. In such anembodiment, demosaicing circuit 1706, for each pixel of the Bayerpattern color filter array image data frame 5204, interpolates red,green, and blue values. Referring to frame 5204 shown in FIG. 16 b,optical reader 100 determines a red value for red pixel position P₃₂simply by reading the color scale value of pixel position P₃₂. Opticalreader 100 determines a green value for red pixel P₃₂ by averaging thevalues of green pixel positions P₃₁, P₂₂, P₄₂ and P₃₃. Optical reader100 may interpolate a blue value for red pixel position P₃₂ by averagingthe values of blue pixel positions P₁₄ P₄₁, P₂₃, P₄₃. It will be seenthat red, green, and blue values can be determined for each pixelposition interpolating the pixel values as necessary. With increasedprocessing speeds, it is possible to utilize dozens or more surroundingpixel values for interpolation of a red, green, or blue pixel for eachpixel position.

In another aspect of the invention, the accuracy with which color scalevalues for each pixel position may be interpolated can be enhanced byutilizing monochrome pixel values in the color scale value interpolationprocess. Referring to red pixel position P₃₂ of frame 5204, it has beendescribed that color scale values at green pixel positions P₃₁, P₂₂,P₄₂, P₃₃ may be averaged for interpolating a green color scale value atpixel position P₃₂. In another method, monochrome pixel values atpositions P₃₃, P₂₂, P₄₂, P₃₃ may be utilized for enhancing theinterpolation of a green pixel value at position P₃₂. The monochromepixel values at positions P₃₃, P₂₂, P₄₂, P₃₃ may be interpolated frommonochrome pixel values by one of the monochrome pixel interpolationmethods described herein. Then, the color scale value at each pixelposition, P₃₂, P₂₂, P₄₂, P₃₃ may be offset by a value, Delta, equal tothe difference between the interpolated monochrome pixel values at theposition being interpolated and the monochrome pixel value at theposition contributing to the interpolation calculation. Thus, a greencolor scale value at position P₃₂ may be calculated according to Eq. A.

$\begin{matrix}{{G\left( P_{32}^{\;} \right)} = \frac{\begin{matrix}{\left\lbrack {{G\left( P_{31} \right)} + {Delta}_{31}} \right\rbrack + \left\lbrack {{G\left( P_{22} \right)} + {Delta}_{22}} \right\rbrack +} \\{\left\lbrack {{G\left( P_{42} \right)} + {Delta}_{42}} \right\rbrack + \left\lbrack {{G\left( P_{33} \right)} + {Delta}_{33}} \right\rbrack}\end{matrix}}{4}} & \left( {{Eq}.\mspace{14mu} A} \right)\end{matrix}$

Where Delta₃₁=M(P₃₂)−M(P₃₁), Delta₂₂=M(P₃₂)−M(P₂₂),Delta₄₂=M(P₃₂)−M(P₄₂), Delta₃₃=M(P₃₂−M(P₃₃).

Similarly, a blue color scale value at position P₄₂ may be interpolatedaccording to the formula of Equation B.

$\begin{matrix}{{B\left( P_{42}^{\;} \right)} = \frac{\left\lbrack {{B\left( P_{41} \right)} + {Delta}_{41}} \right\rbrack + \left\lbrack {{B\left( P_{43} \right)} + {Delta}_{43}} \right\rbrack}{2}} & \left( {{Eq}.\mspace{14mu} B} \right)\end{matrix}$

Where Delta₄₁=M(P₄₂)−M(P₄₁) and Delta₄₃=M(P₄₂)−M(P₄₃).

An exemplary algorithm for interpolating a color scale value at a colorpixel position utilizing monochrome pixel values is presented in Table 4where “C-set” refers to color pixel values and “M-set” refers tomonochrome pixel values.

TABLE 4 Algorithm For Interpolating Color Scale Values UtilizingMonochrome Image Data 1) For each color pixel C for interpolation,select the missing color neighborhood C-set pixel values Ci and selectthe corresponding neighborhood M-set pixel values Mi. Selectcorresponding M-set pixel value M to color pixel C. 2) let C=0 3) fori=1 to n where n is the number of neighborhood pixel Ci 4) C=C+Ci+M−Mi5) end 6) C=C/n

Regarding step 1, it is noted that there will normally be twoneighborhood color or “C-set” pixels where blue or red values areinterpolated at a green pixel position, and in other cases fourneighborhood color pixels.

Another particular example of optical reader 100 executing steps 1405and 1406 is explained with reference to FIG. 16 c. Where a reader imagesensor array is provided by image sensor array 182B including 1280×1024array of pixels, and a period P=4 of color sensitive pixels formed witha Cyan-Magenta (Cy-Mg, or “CM”) color filter array as shown in FIG. 5 b,optical reader 100 at step 1405 reads out a color filter array frame5206 as shown in FIG. 16 c. Color filter array image data frame 5206includes a 320×256 pattern of Cy-Mg pixel values. Demosaicing circuit1706 may process image data frame 5206 into a visual display frame suchas a visual display color frame of image data where each pixel positionof frame 5206 is represented by a combination of red, green and bluevalues. In processing the Cy-Mg visual display color frame 5206 into aframe of image data including red, green, and blue values for each pixelposition, optical reader 100 may first calculate white, cyan and magentavalues for each pixel position of frame 5206. Where an original pixelposition such as pixel position P₅₃ (FIG. 16 c) is a cyan pixel, thecyan value is determined by directly reading the pixel value of the cyanpixel. A magenta value for cyan pixel at position P₅₃ is calculated byinterpolation utilizing the magenta values of surrounding pixelpositions of magenta pixels such as positions P₅₂, P₄₃, P₆₃, P₅₄ (FIG.16 c). A white value for cyan pixel at position P₃₅ is calculated byinterpolation using pixel values from monochrome pixel positions thatsurround cyan pixel P₅₃. Referring to FIG. 5 b, a supplemental frameincluding monochrome pixel values may be captured, e.g., successivelybefore or after frame 5206 is captured for purposes of interpolating awhite value for each color pixel of the color filter array windowedframe 5206. Alternatively, the color filter array frame 5206 captured atstep 1405 may include monochrome pixel image data for purposes ofinterpolating a white value for each color pixel value. When white, cyanand magenta values are calculated for each pixel of frame 5206, thewhite, cyan, and magenta values are readily converted into red, green,and blue values. Alternatively, display 504 can be configured to beresponsive to white, cyan and magenta signals for each pixel of display504. A set of transform equations for transforming a set of white, cyanand magenta values for a given pixel of a frame into a set of red, greenand blue values for that pixel is given as follows.

R=W−Cy  (Eq. 1)

G=Mg+Cy−W  (Eq. 2)

B=W−Mg  (Eq. 3)

In the process described relative to the flow diagram of FIG. 14 g, anoriginal color filter array frame is processed into a visual displaycolor frame of image data of reduced spatial resolution (a reducedspatial resolution 320×256 visual display color frame of image data maybe produced using a hybrid monochrome and color image sensor arrayhaving a 1280×1024 pixel resolution). With reference to FIG. 14 h, aprocess for producing a high spatial resolution visual display colorimage is described. In the process described relative to the flowdiagram of FIG. 14 h, optical reader 100 utilizes image data from bothmonochrome pixels 250M and color pixels 250C from a hybrid monochromeand color image sensor array such as image sensor array 182A or imagesensor array 182B in the generation of a visual display color imagehaving spatial resolution equal to or on the order of the overall pixelresolution of the image sensor array.

At step 1408 control circuit 552 captures a color filter array imagedata frame by selectively addressing color pixels 250C of an imagesensor array and selectively reading out image data from color sensitivepixels 250M. The frame of image data captured at step 1408 is a windowedframe of image data. For reduction of cross talk resulting from lightentering monochrome pixels 250M, the monochrome pixels of image sensorarray 182A, 182B may be reset using reset control timing pulse 370,370′, 370″, 370″′ at the time that exposure control timing pulse 354,354′, 354″, 354″′ is applied to expose color pixels for capture of acolor filter pattern image frame at step 1408.

At step 1409 control circuit 552 captures a monochrome frame of imagedata by selectively addressing monochrome pixels 250M of array 182A,182B and selectively reading out image data from monochrome 280M pixels.The frame of image data captured at step 1409 is a windowed frame ofimage data. For reduction of cross-talk resulting from light enteringcolor pixels 250C the color pixels of image sensor array 182 may bereset using reset control timing pulse 370, 370′, 370″, 370′″ at thetime that exposure control timing pulse 354, 354′, 354″, 354′″ isapplied to expose monochrome pixels for capture of a monochrome,typically gray scale or binarized image frame at step 1409.

At step 1410 control circuit 552 transfers both the color filter arrayframe captured at step 1408 and the monochrome image frame captured atstep 1409 to fusion circuit 1708. Fusion circuit 1708 takes as inputsthe color filter array image data frame and the monochrome image dataframe and processes them into a high resolution visual display colorframe of image data.

Referring to FIG. 14 i, an example of the process 1440 which may beexecuted by fusion circuit 1708 (FIG. 1 e) to process a combination ofmonochrome image data and color image data into a visual display colorframe of image data is described. As explained with reference to FIG. 1e, fusion circuit 1708 may be physically embodied by the combination ofa control circuit provided by a CPU 552 operating in combination withmemory 566 that stores an executable program. The specific processdescribed with reference to FIG. 14 i is executed utilizing an opticalreader 100 including substantially uniform dimensional pixel imagesensor array 182B. At step 1442 of process 1440 control circuit 552generates color filter array image data and monochrome gray scale imagedata. Where optical reader 100 includes image sensor array 182B, controlcircuit 552 may execute step 1442 by reading out from image sensor array182B a single frame of image data comprising both monochrome image dataand color image data. Control circuit 552 may also execute step 1442 bysuccessively capturing a first monochrome frame comprising monochromeimage data and then a second color frame comprising color image data.Control circuit 552 at step 1442 may drive monochrome pixels 250M intoreset during an exposure period for read out of color image data fromcolor sensitive pixels 250C. When generating a frame of monochrome imagedata at step 1442, control circuit may interpolate monochrome pixelvalues for “missing pixel” positions occupied by color sensitive pixels250C.

At step 1446 control circuit 552 generates an RGB image havingresolution equal to the color sensitive subset of pixels of image sensorarray 182B. In an RGB image, each pixel of the image is represented by ared color value, a green color value and a blue color value. The RGBimage generated at step 1446 may have the same characteristics as thevisual display image received by optical reader 100 at step 1407 of thealternative process described in connection with FIG. 14 g. Where acolor filter array image captured at step 1442 is a Bayer pattern image,the RGB image generated at step 1446 is derived by executing ademosaicing routine as described herein. Where the color filter arrayimage captured at step 1442 is a CMY image or a CM image (cyan andmagenta only) image as described in connection with FIGS. 2 b and 2 d,the RGB image generated at step 1446 is derived by way of atransformation process as described herein in connection with equations1, 2 and 3. With further reference to process 1440 which may be executedby fusion circuit 1708 control circuit 552 at step 1450 expands thepixel count of the RGB image generated at step 1446 so that the pixelcount of the color image is equal to the pixel count of the monochromeimage captured at step 1442 (at step 1442 monochrome pixels from thecaptured monochrome image may be interpolated as described with FIG. 16a). When control circuit 552 executes step 1450, the monochrome grayscale image generated at step 1442 and the color image at that stage ofthe processing have equal numbers of pixels such that each pixelposition e.g., pixel of the monochrome image has a corresponding pixelposition in the color image. With reference to image sensor array 182Bhaving a period of P=2, there are four times as many monochrome pixelsas there are color sensitive pixels. Accordingly, with image sensorarray 182B having a period of P=2, control circuit 552 at step 1450expands each pixel into a 2×2 pixel block. Where image sensor array 182Bhas a period P=3, control circuit 552 at step 1450 expands each pixelinto a 3×3 pixel block. Where image sensor array 182B includes theperiod of P=4, control circuit 552 at step 1450 expands each pixel intoa 4×4 pixel pixel block. At step 1454 control circuit 552 calculates anintensity value I_(c) for each pixel position of the expanded colorimage. Control circuit 552 at step 1454 calculates an intensity valuefor each pixel position of the expanded color image according to theformula

I _(c)=0.299R+0.587G+0.144B  (Eq. 4)

Control circuit 552 at step 1460 then calculates an intensity valuedelta, D, for each pixel position, (Px, Py) utilizing a monochrome imageintensity value I_(m) and an expanded image color intensity value,I_(c), at each pixel position. Control circuit 552 at step 1460 maycalculate an intensity value delta for each pixel position of themonochrome and expanded color image according to the formula

D(P _(x) ,P _(y))=I _(m)(P _(g) ,P _(y))−I _(c)(P _(g) ,P _(y))  (Eq. 5)

At step 1464, control circuit 552 updates the RGB data set color scalevalues of the expanded RGB color image using the set of formulas

R′(P _(x) ,P _(y))=R(P _(g) ,P _(y))+D(P _(g) ,P _(y))  (Eq. 6)

G′(P _(g) ,P _(y))=G(P _(g) ,P _(y))+D(P _(x) ,P _(y))  (Eq. 7)

B′(P _(g) ,P _(y))=B(P _(x) ,P _(y))+D(P _(x) ,P _(y))  (Eq. 8)

At step 1468 control circuit 552 truncates RGB data set color scalevalues that are greater than 255 (where an 8 bit gray scale is used).After control circuit 552 truncates RGB values greater than 255, controlcircuit 552 at step 1770 outputs a visual display color frame of imagedata having a spatial resolution equal to or approximately equal to theoverall pixel resolution of image sensor array 182B. The visual displaycolor frame of image data output at step 1770 may have a number of RGBdata sets equal to the overall pixel count (e.g., monochrome pixels pluscolor sensitive pixels) of image sensor array 182B.

At step 1411 optical reader 100 receives from fusion circuit 1708 a highresolution visual display color frame of image data. The visual displaycolor frame of image data received at step 1411 may include a pixelresolution equal to or on the order of the pixel resolution of imagesensor array 182B. Optical reader 100 may be regarded as having receiveda visual display color frame of image data when fusion circuit 1708outputs a visual display color frame of image data at step 1470.

When executing process 1440, control circuit 552 fuses monochrome andcolor image data to produce a high resolution visual display color frameof image data. When executing the alternative process described withreference to the flow diagram of FIG. 14 j, control circuit 552 fusesmonochrome and color image data in such a manner that color reproductionis optimized.

In general, increasing the percentage of monochrome pixels 250M in imagesensor array 182A, 182B increases indicia decoding accuracy, whileincreasing the percentage distribution of color sensitive pixels 250C inimage sensor array increases color reproduction accuracy. Because of thelight transmissivity of monochrome pixels, an image obtained utilizingan image sensor array having a higher percentage of monochrome pixels250M has a higher signal to noise ratio than an image obtained utilizingan image sensor array having a smaller percentage of monochrome pixels250M. Accordingly, an image obtained utilizing an image sensor arrayhaving a higher percentage of monochrome pixels often produces an imagewith greater detail and improved overall visual quality.

Optical reader 100 in another aspect may incorporate the structure shownin FIG. 21. In FIG. 21, center pixels 2072 of reader image sensor array182B have a higher percentage of monochrome pixels 250M, i.e., a periodof P=4, as shown in FIG. 5 e, while outer pixels 2074 have a lowerpercentage of monochrome pixels 250M, i.e., a period of P=2, as shownand described in FIG. 5 c. The image sensor array 182B is constructedsuch that center pixels 2072 are optimized for providing image datayielding increased decoding accuracy while outer pixels 2074 areoptimized for providing image data yielding increased color reproductionaccuracy.

Referring to further steps of process 1400, control circuit 552 at step1412 outputs the visual display color frame of image data obtained atstep 1404. At step 1412 control circuit 552 may output a visual displaycolor frame of image data to display 504 for visual observation by anoperator or to a designated color frame storage memory location ofreader 100 such as a designated frame memory storage location of Flashmemory 564 or to another frame memory location of system 145. Wherecontrol circuit 552 is incorporated in hand held housing 101, controlcircuit 552 at step 1410 may also send a visual display color frame ofimage data to spaced apart device 150, as shown in FIG. 10. For sendinga frame of image data to a spaced apart device 150, optical reader 100,the spaced apart device 150, and a communication link there between maybe configured to transmit data packets in accordance with a protocol ofthe TCP/IP suite of protocols. Further, optical reader 100 may formatthe visual display color frame of image data obtained at step 1412 in asuitable image file format (e.g., .BMP, .TIFF, .PDF, .JPG, .GIF) andoptical reader 100 may automatically send the visual display color frameof image data at step 1412 utilizing the file transfer protocol (FTP).Optical reader 100 at output step 1212 may format the visual displaycolor frame of image data in a suitable image file format (e.g., .BMP,.TIFF, .PDF, .JPG, .GIF) when storing the visual display color frame ofimage data in memory 566 (which can be incorporated in hand held housing101) or when sending the visual display color frame of image data to aspaced apart device 150 for storage. Optical reader 100 may alsotransmit a visual display color frame of image data utilizing a suitablemarkup language such as .XML. Referring to FIG. 10, system 145 may beconfigured so that when a display equipped spaced apart device 150receives a visual display color frame of image data from optical reader100, the spaced apart device 150 automatically displays that receivedvisual display color frame of image data on a display 1504 associatedwith that device.

Optical reader 100 can be configured so that all the steps of process1400 are carried out automatically in response to receipt of a triggersignal until a stop condition is satisfied. A stop condition may be thereceipt of a trigger stop signal such as may be generated by the releaseof trigger 216.

In the embodiments of FIGS. 14 a-14 c, two actuations of a readercontrol button are made to carry out an indicia decode process and twoactuations of a reader control button are made to carry out a picturetaking process (one actuation of button 3162 or button 3164 to configurethe reader 100 and then another actuation of trigger 216 to capture animage). It will be understood that optical reader 100 can be configuredto carry out indicia decoding or picture taking with a single actuationof a reader control button. For example, optical reader 100 can beconfigured so that actuation of virtual button 3162 both configuresreader 100 to decode and simultaneously generates a trigger signal toimmediately commence image capture and decoding. Optical reader 100 canalso be configured so that actuation of virtual icon button 3164 bothconfigures a reader 100 for picture taking and simultaneously generatesa trigger signal to immediately commence image capture.

While process 1200 and process 1400 may be carried out in thealternative, process 1200 and process 1400 may also be executedcontemporaneously. For example, while control circuit 552 obtains adecode frame at step 1204, control circuit 552 may obtain a visualdisplay color frame of image data at step 1404. Control circuit 552 mayobtain a color frame of image data as a decode frame at step 1204 andthen outputs that frame at step 1212 as visual display color frame ofimage data. Control circuit 552 at step 1412 may output a visual displaycolor frame of image data and contemporaneously transfer that frame ofimage data to decode circuit 1702. In general, reader 100 may beconfigured so that whenever control circuit 552 obtains a decode frameat step 1204, control circuit 552 may store that frame for laterprocessing, which processing may include processing for generating avisual display color frame of image data and which processing may beresponsive to an operator input command to perform such processing.Optical reader 100 may also be configured so that when control circuit552 obtains a visual display color frame of image data at step 1404,control circuit may store that frame for further processing, whichprocessing may include transferring that frame to decode circuit 1702 orautodiscrimination circuit 1704, and which processing may be responsiveto an operator input command to perform such processing.

Another embodiment of the invention is described with reference to FIGS.17 a-17 g. In the embodiment of FIGS. 17 a-17 g optical reader 100includes a pair of imaging modules 1802D and 1802E. Imaging module 1802Dis a color imaging module having color image sensor array 182D. Colorimage sensor array 182D includes a Bayer pattern color filter with oneof red, green or blue wavelength selective filter disposed on eachpixel. Imaging module 1802E as shown in FIG. 17 e is a monochromeimaging module having a one-dimensional solid state image sensor array182E. One-dimensional monochrome image sensor array 182E in theembodiment of FIGS. 17 a, 17 e, 17 f, and 17 g includes an M×1 (one row)array of monochrome (without color filter) pixels. One-dimensional imagesensor array 182E may also include and M×N array of pixels, where M>>N,e.g., an M×2 (2 rows) of pixels.

The reader 100 shown in the electrical block diagram of FIG. 17 a hasmany of the same components as shown in optical reader 100 of FIG. 1 a.Namely, optical reader 100 of FIG. 17 a includes a control circuit 552provided in the example by a CPU, which operates under the control ofprogram data stored in EPROM 562. Control circuit 552 is incommunication with a memory unit 566 that includes in addition to EPROM562, RAM 560, and Flash memory 564. Control circuit 552 further receivesinput control data from various user input devices such as manualtrigger 216, pointer controller 512, keyboard 508 and touch screen 504T.Control circuit 552 may also output data such as decoded output data andvisual display image data to color display 504. For capturing imagedata, control circuit 552 may control either image sensor array 182E orimage sensor array 182D. For capturing one-dimensional image datacorresponding to a one-dimensional slice image of a target, controlcircuit 552 sends various image capture initiating control signals toone-dimensional image sensor array 182E. In response to the imagecapture initiation control signals, image sensor array 182E sends analogimage signals to signal processing circuit 591 which among variousprocessing functions amplifies the signals and feeds the signals toanalog-to-digital converter 592. Analog-to-digital converter 592converts the signals into digital form and routes the digital image datato FPGA 593 which under the control of control circuit 552, manages thetransfer of the digital information into RAM 560, where the monochromeimage data can be accessed for decoding processing by control circuit552. For capturing two-dimensional frames of color image data, controlcircuit 552 sends appropriate image capture initiation control signals(e.g., exposure, read out) to image sensor chip 1082. FPGA 580 receivesdigital image data from image sensor IC chip 1082, 1082D and under thecontrol of control circuit 552 manages the transfer of color image datainto RAM 560. Illumination assembly 104 for each module 1802D, 1802E maybe controlled during image acquisition as explained with reference tothe timing diagrams of FIGS. 15 a-15 e.

Optical reader 100 as shown in FIGS. 17 a-17 g may be operated inaccordance with the flow diagram of FIGS. 17 a-17 g. Namely, by asuitable selection method such as by depressing icon 3162 or icon 3164(FIG. 9 b) one of a decode mode of operation and a color image capturemode of operation can be selected. However, in the dual imaging moduleembodiment of FIGS. 17 a-17 g, the imaging module which is utilized forcapturing image data depends on which mode (indicia decoding, or picturetaking) is selected. If the indicia decode mode is selected at step 1100(FIG. 14 a) and a trigger signal is received, optical reader 100proceeds to step 1102 to execute indicia decode process 1200 (FIG. 14a). At step 1204 of indicia decode process 1200, control circuit 552obtains a decode frame of image data. If the picture taking mode ofoperation is selected at step 1100 (FIG. 14 a) and a trigger signal isreceived, control circuit 552 proceeds to step 1404 (FIG. 14 c) toobtain a visual display color frame of image data. Where reader 100includes two imaging modules, one color such as module 1802, 1802Dhaving color image sensor array 182, 182D and one monochrome such asmodule 1802, 1802E having monochrome image sensor 182, 182E, theparticular image sensor array 182 to which control circuit 552 sendscontrol signals for initiating image capture depends on whether opticalreader 100 is operating in a decode mode of operation or a picturetaking mode of operation. With reference to the reader 100 of FIGS. 17a-17 g, and the flow diagrams of FIGS. 14 a, 14 b and 14 c, reader 100at step 1204 sends image capture initiation control signals tomonochrome one-dimensional image sensor array 182, 182E to initiateimage capture without sending any image capture initiation controlsignals to color image sensor array 182, 182D if the reader 100 isoperating in a decode mode operation. Reader 100 at step 1404 sendsimage capture initiation control signals to color image sensor array182, 182D without sending any image capture initiation control signalsto monochrome image sensor array 182, 182E if reader 100 is operating ina picture taking mode operation. Accordingly, where optical reader 100is in an indicia decode mode and receives a trigger signal, a monochromeframe of image data is sent to RAM 560 for further processing by decodecircuit 1702 (FIG. 10). Where optical reader 100 is in a picture takingmode and receives a control signal, a color image is sent to RAM 560.The color image if a Bayer pattern image is subject to a demosaicingprocess as described herein for generating a visual display color frameof image data, which visual display color frame of image data may beoutput by control circuit 552 e.g., to display 504 and/or a designatedmemory address of system 145 (e.g., memory 566 or another memory such asa memory of a spaced apart device 150), and/or to a display 1504 of aspaced apart device 150 of system 145 (FIG. 5).

When an image is captured by the two imaging module readers of FIGS. 17a-17 g, the type of image capture (monochrome or color) depends on aselected operating mode. When an indicia decode mode is selected, amonochrome gray scale image well suited for decode processing iscaptured. When a picture taking mode is selected, a color image iscaptured which is well suited for visual display.

Further aspects of a dual imaging module reader are described withreference to FIGS. 17 b-17 g. FIGS. 17 b and 17 c illustrate that thehardware block 598 of reader 100 shown in FIGS. 17 a, 17 f, and 17 g maybe replaced with alternative hardware blocks. As indicated by FIG. 17 b,hardware block 398 which in FIG. 17 a includes a CCD one-dimensionalsolid state image sensor array 182E and off-board signal processingcircuit 591, analog-to-digital converter 592 and FPGA 593 may bereplaced by a hardware block including an CMOS image sensor IC chip1082F including a monochrome image sensor array 182F. Image sensor ICchip 1082, 1082F is of construction similar to image sensor IC chip1082, 1082A and IC chip 1082, 1082D except that image sensor array 182Fof chip 1082F includes monochrome pixels 250, 250M only and is devoid ofcolor sensitive pixels 250, 250C. FIG. 17 c illustrates that imagingassembly hardware block 598 can be replaced with a laser scanning barcode engine 594 and an associated decode circuit 595. Laser scanning barcode engine 594 and associated decode circuit 595 may be available in apackage known as an SE 923 decoded out scan engine available from SymbolTechnologies. In the embodiment of FIG. 17 c, steps 1210, 1212, 1214 ofdecode process 1200 are carried out by decode circuit 595.

Exemplary imaging modules supporting various types of image sensor ICchips are shown in FIGS. 17 d and 17 e. FIG. 17 d shows an exemplaryimaging module for supporting image sensor IC chip 182D. Imaging module1082D includes the elements shown and described with reference to FIGS.8 a-8 d except that imaging module 1082D includes image sensor IC chip182D and further except that certain light sources are optionallydeleted. Imaging module 1082E includes the elements shown and describedwith reference to FIGS. 8 a-8 e except that imaging module 1082Eincludes one-dimensional monochrome image sensor chip 182E and furtherexcept that certain light sources of illumination block 104 areoptionally deleted. With module 1802E, aiming pattern 1838 (FIG. 8 d)may serve as an aiming and illumination pattern. Further, it will benoted that illumination assembly 104 of an imaging module herein mayinclude a flash illumination light source, 160, 160X (FIG. 9 a). It maybe particularly useful to incorporate a flash illumination intoillumination assembly 104, where an imaging module 1082 is usedprimarily for capture of visual display color image.

Referring to FIGS. 17 f and 17 g construction views of dual imagingmodule readers incorporated in various optical reader housings are shownand described. In FIG. 17 f a gun style optical reader 100 is shownhaving color two-dimensional imaging module 1802D and one-dimensionalmonochrome imaging module 1082E supported therein. In FIG. 17 g aportable data terminal (PDT) optical reader 100 is shown having colortwo-dimensional imaging module 1802D and one-dimensional monochromeimaging module 1802E supported therein. The dual modules can also beinstalled in other types of housings such as cell phone housings (FIG. 9c) and personal data assistant housings (PDAs). In the examples of FIGS.17 f and 17 g, imaging modules 1802 are supported by struts 597 formedon an interior wall 1802 of housing 101. Via ribbon connectors 598, themodules 1802 in each example are in communication with a main printedcircuit board 599 which includes various electrical components includingprocessor IC chip 548.

In one application, the optical reader 100 of FIGS. 17 a-17 g isoperated in the following manner. An operator actuates color imagesensor array 182D to take a color picture of parcel 1260 (FIGS. 11 and12) carrying a bar code symbol 1266, 1270. Such actuation may be carriedout, e.g., by depressing decode button 3164 and then trigger 216 orbutton 3164 only. An operator then actuates monochrome image sensorarray 182E (or alternatively image sensor array 182F, or laser scanengine 594) to decode bar code symbol 1266, 1270. Such actuation may becarried out e.g., by depressing button 3162 and then trigger 216 or bydepressing button 3162 only. Further, control circuit 552, which may beincorporated in hand held housing 101, may transmit a visual displaycolor frame of image data representing parcel 1260 and decoded outmessages corresponding to one or more of symbols 1266, 1270 to remoteserver 184 (FIG. 10). System 145 can be configured so that suchtransmission is automatic in response to trigger signals being received,or optical reader 100 can be configured so that associated color picturedata and decoded out bar code message data are transmitted in responseto receipt of a user-initiated command input into a user-interface ofoptical reader 100 to transmit associated picture and decoded bar codemessage data.

With further reference to the reader electrical block diagram shown inFIG. 1 a, various useful optical reader embodiments may be yielded byreconfiguration of hardware block 208 including an image sensor array.With reference to FIG. 18 a, an optical reader 100 having the hardwarecomponents shown in FIG. 1 a may be modified to include an image sensorarray 182C as shown and described in connection with FIG. 18 a. In theembodiment of FIG. 18 a, optical reader 100 includes cyan-magenta-yellow(CMY) color filter array 182C. Each pixel 250 of image sensor array 182Cincludes a color filter element; namely one of a cyan color filterelement, a magenta color filter element or a yellow color filterelement. Yellow color filter elements have excellent light transmittance(approaching the transmittance of a monochrome pixel). Further, it isseen that in accordance with the CMY color filter pattern shown in FIG.18 a that approximately 50% of all pixels of image sensor array 182C areyellow pixels (pixels having a yellow light wavelength sensitive filterelement). In the specific example of FIG. 18 a, image sensor array 182Chaving cyan, magenta and yellow pixels is devoid of green pixels.However, image sensor arrays are available which have green pixels inaddition to cyan, magenta and yellow pixels. Image sensor array 182C maybe incorporated into an optical reader 100 that operates in accordancewith the picture taking mode/indicia decode mode flow diagram describedin connection with FIG. 14 a. That is, when driven into a indicia decodemode of operation as described in connection with FIG. 14 b, opticalreader 100 including CMY color image sensor array 182C obtains a decodeframe of image data whereas when optical reader 100 including imagesensor array 182C is driven into a picture taking mode of operation,optical reader 100 obtains a visual display color image frame of imagedata as described in connection with FIG. 14 c herein.

According to the invention, an optical reader including a CMY imagesensor array 182C as shown in FIG. 18 a may obtain image data in amanner that depends on which operational mode (indicia code or picturetaking) is selected. Where optical reader 100 including CMY image sensorarray 182C obtains a decode frame of image data at step 1204, controlcircuit 552 of optical reader 100 can selectively address yellow colorpixels of CMY image sensor array 182C and selectively read out imagedata only from yellow colored pixels of image sensor array 182C. Withfurther reference to a reader including image sensor array 182C, controlcircuit 552 at step 1204 may interpolate missing pixel valuescorresponding to the pixel positions of magenta and cyan pixels of imagesensor array 182C. After interpolating the missing pixel positions,control circuit 552 at step 1210 may transfer the interpolated decodeframe to one of indicia decode circuit 1702 or autodiscriminationcircuit 1704.

In a further aspect of the optical reader described in connection withFIG. 18 a including a CMY color image sensor array 182C, image sensorarray 182C may include separate and independent reset control lines forfacilitating the reset of magenta (labeled “Mg”) and cyan (labeled “Cy”)pixels independent from the resetting of yellow pixels (labeled “Y”).Accordingly, when image data at step 1204 is read out selectively fromyellow pixels, the magenta and cyan pixels of image sensor array 182Cmay be driven into reset to eliminate electron diffusion cross talk andto reduce cross talk attributable to photons entering image sensor array182C through magenta and cyan color pixels 250C.

When obtaining a visual display color frame of image data as describedat step 1404 of the flow diagram FIG. 14 c an optical reader includingimage sensor array 182C may simply read out image data from all of thepixels of the array 182C and execute a simple demosaicing algorithm toconvert a single color value for each pixel of image sensor array 182Cinto a visual display color image wherein each pixel of image sensorarray 182C is represented by a data set including three color scalevalues, e.g., a cyan color scale value, a magenta color scale value anda yellow color scale value.

Control circuit 552 at step 1404 where the reader includes a CMY imagesensor array 182C may transform the CMY visual display image into an RGBvisual display image utilizing a CMY to RGB transformation process asdescribed herein.

The performance of optical reader 100 may be hindered where opticalreader 100 is operated to read bar code symbols or other indiciadisposed on a substrate having a shiny surface (e.g., metal, glass,laminated, plastic, etc.). Light rays emanating from light sources 160of reader 100 that are projected on a highly reflective shiny surface ofa substrate, s, may be substantially entirely reflected directly on toimage sensor array 182. “Specular” reflection is said to occur where asubstantial percentage of light rays are reflected and directed ontoimage sensor array 182. Light rays are said to be reflected at a“specular angle” when light rays are reflected from a surface at aboutthe angle of incidence. Specular reflection tends to saturate imagesensor array 182 to cause decoding failures. The optical reader 100described in connection with FIGS. 19 a-c is configured so that readerrors resulting from specular reflection are reduced. As shown anddescribed in connection with FIG. 19 a, hardware block 208 shown in FIG.1 a as including a hybrid monochrome in color image sensor array 182Acan be replaced with hardware block 208 as shown in FIG. 19 a includinga hybrid monochrome and polarizer filter image sensor array 182G.

Image sensor array 182G includes a first subset of monochrome pixels250M and a second subset of light polarizing pixels 250P. Lightpolarizing pixels 250P of image sensor array 182G include lightpolarizing filter elements 261 (alternatively termed “light polarizingfilters,” or simply “light polarizers”) typically formed at eachpolarizing pixel 250P in the position of filter 260 as shown in thecolor pixel views of FIGS. 3 c and 6 c. Light polarizing filter elements261 of image sensor array 182G, 182H can be deposited onto the majorbody of light polarizing pixels 250P by way of a depositing process.Light polarizing filter elements 261 of image sensor array 182G can beconstructed to attenuate polarized light rays generated from anappropriately polarized light source and reflected at a specular angle.Accordingly, polarized light rays incident on the image sensor array onthe polarizing pixels 250P are attenuated significantly; thus, reducingthe contribution of specularly reflected light rays to generated imagesignals from the polarizing pixels 250P.

According to the invention, optical reader 100 including image sensorarray 182G may be configured to selectively address light polarizingpixels 250P and selectively read out image data from light polarizingpixels 250P to generate image data for subjecting to decoding which islikely to result in successful reading of bar codes or other indicianotwithstanding the image data being obtained during specularreflections read conditions.

Referring to FIG. 19 b, a perspective view of light polarizing imagesensor array 182G is shown with an exploded view showing a pattern whichmay be repeated throughout the array. In the version of FIG. 19 b, lightpolarizing pixels 250P having light polarizing light filter elements 261are uniformly distributed throughout image sensor array 182G with aperiod of P=2. It will be understood that light polarizing pixels 250Pmay also be distributed throughout image sensor array 182G in a uniformor substantially uniform distribution pattern other than the patternshown in FIG. 19 b. For example, light polarizing pixels 250P may bedistributed throughout image sensor array 182G with a distributionpattern of P=3, (as described in connection with FIG. 5 d showing ahybrid monochrome and color image sensor array) or a distributionpattern having the period of P=4, as illustrated with reference tohybrid monochrome and color image sensor array as shown in FIG. 5 e.

Referring to the view of FIG. 9 b, optical reader 100 may be operated ina mode in which optical reader 100 captures image data by selectivelyaddressing polarizing pixels 250P and selectively reading out image datafrom light polarizing pixels 250P only. Optical reader 100 may beconfigured to have a reduced specular reflection read error decode mode.Optical reader 100 can be configured so that when button 3156 isactuated, optical reader 100 receives a trigger signal to obtain imagedata that is likely to result in successful reading notwithstandingspecular reflection reading conditions.

Referring to the flow diagram of FIG. 19 c, optical reader 100 at step1902 may receive a trigger signal to commence operation in a reducedspecular reflection read error decode mode. The trigger signal may bereceived pursuant to a manual control by an operator such as anactuation of control button 3156. Control circuit 552 may also beconfigured to receive the trigger signal at step 1902 when controlcircuit 552 automatically senses a predetermined condition such as asaturation condition. Control circuit 552 at step 1902 may determinethat a saturation condition is present by analysis of image data at step1204 (FIG. 14 b) during normal decoding operations so that when asaturation condition is detected, optical reader 100 automaticallycommences operation in a reduced specular reflection read error decodemode. In a specific embodiment of the invention, control circuit 552 maydetermine that a saturation condition is present when an average whitevalue of monochrome image data is below a predetermined level.

At step 1904 optical reader 100 obtains a specular reflection readcondition decode frame of image data. Control circuit 552 obtains aspecular reflection condition decode frame of image data at step 1902 byselectively addressing light polarizing pixels 250P of image sensorarray 182G and selectively reading out image data from light polarizingpixels 250P only. In another aspect of image sensor array 182G that maybe incorporated in optical reader 100, image sensor array 182G mayinclude separate reset control lines for resetting monochrome pixels250M separately and independently of light polarizing pixels 250P. Imagesensor array 182G may have separate sets of reset control lines asdescribed in connection with image sensor array 182G, particularly inconnection with FIG. 7 a.

Accordingly, when control circuit 552 selectively addresses lightpolarizing pixels 250P for read out of image data from light polarizingpixels 250P, control circuit 552 drives monochrome pixels 250M intoreset. Resetting of monochrome pixels 250M is synchronized with theexposure period for exposing light polarizing pixels 250P as describedherein. Driving monochrome pixels 250M into reset while light polarizingpixels 250P are exposed eliminates electron diffusion cross talk andreduces cross talk resulting from photon penetration to image sensorarray 182G.

At step 1904, control circuit 552 may interpolate pixel values at pixelpositions corresponding to missing pixel positions. At step 1906 controlcircuit 552 transfers the specular reflection condition decode frame ofimage data obtained at step 1904 to indicia decode circuits 1702 orautodiscrimination circuit 1704 as are described in connection with FIG.1 e.

At step 1908 control circuit 552 receives decoded output data output bydecode circuit 1702 or signature autodiscrimination circuit 1704. Atstep 1910 control circuit 552 outputs decoded out data, e.g., bytransferring decoded out data to an on reader display 504 or to a spacedapart display 1504 or else stores decoded data in appropriate memoryaddress location of system 145 (FIG. 10).

A process has been described with reference to the flow diagram of FIG.19 c wherein control circuit 552 selectively reads out monochrome pixelimage data from monochrome pixels 250M and selectively reads out imagedata from light polarizing pixels 250P. An optical reader includinghybrid monochrome and light polarizing image sensor array 182G may alsobe operated without selectively reading out image data from image sensorarray 182G. An optical reader incorporating hybrid monochrome and lightpolarizing image sensor array 182G can be operated to decode decodableindicia and to take pictures in accordance with the process describedwith reference to the flow diagrams of FIGS. 14 a, 14 b, and 14 c. Inobtaining a decode frame of image data (step 1204, FIG. 14 b), controlcircuit 552 may read out image data from all pixels of hybrid monochromeand light polarizing image sensor array 182G including image data fromall monochrome pixels 250M and all light polarizing pixels 250P in asingle frame capture step. The full frame monochrome and light polarizerpixel image data can also be captured with two frame capture steps. Atstep 1210, control circuit 552 may transfer to decode circuit 1702 orautodiscrimination circuit 1704 the full frame of monochrome andpolarized pixel image data obtained at step 1204. If decode circuit 1702or autodiscrimination circuit 1704 fails to decode or fails to detectthe presence of handwritten characters, control circuit 552 may, afterstep 1210, transfer a subset of full frame of image data originallytransferred at step 1210. Namely, after step 1210, if decoding orautodiscrimination fails, control circuit 552 may transfer to decodecircuit 1702, or autodiscrimination circuit 1704 a reduced resolutionimage extracted from a full frame image by selectively extractingmonochrome image data from the full frame of image data. The reducedresolution frame of image data includes only image data corresponding tolight polarizing pixels 250P of image sensor array 182G. The failure ofdecode circuit 1702 to decode or autodiscrimination circuit to recognizemay be regarded as a determination by control circuit 552 that asaturation condition is present.

The elements of a hybrid monochrome and color image sensor array (suchas image sensor array 182A or 182B) as described herein can be combinedwith the elements of a hybrid monochrome and light polarizing imagesensor array 182G into a single image sensor array. FIGS. 20 a and 20 bshow an image sensor array 182H including a first subset of monochromepixels 250M, a second subset of color sensitive pixels 250C and a thirdsubset of light polarizing pixels 250P. Image sensor array 182H mayinclude three separate sets of reset control lines to enable separateand independent of resetting of monochrome pixels 250M, of colorsensitive pixels 250C and of light polarizing pixels 250P. Image sensorarray 182H may be incorporated in hand held optical reader 100 and maybe substituted for hardware block 208 as shown in FIG. 1 a. Opticalreader 100 incorporating image sensor array 182H may have operatingmodes in which optical reader separately addresses monochrome pixels250M for read out of image data from monochrome pixels 250M only.Optical reader 100 including image sensor array 182H may also have anoperating mode in which optical reader 100 selectively addresses colorsensitive pixels 250C and selectively reads out image data from colorsensitive 250C. Optical reader 100 may also have an operating mode inwhich optical reader 100 selectively addresses light polarizing pixels250P and selectively reads out image data from light polarizing pixels250P. Optical reader 100 may obtain a full frame of image data includingmonochrome, color and light polarizing pixels image data (obtained withone, two, or three frame capture steps) and then utilize the image dataon an as needed basis. For example, if a decode attempt utilizing thefull frame image data fails, optical reader 100 may selectively extractlight polarizing pixel image data from the full frame image data andtransfer the extracted image data to decode circuit 1702.

In general, optical reader 100 including image sensor array 182Hselectively reads out image data from monochrome pixels 250M inobtaining a decode frame of image data for transferring to a decodecircuit 1702 under normal read conditions. Optical reader 100selectively reads out image data from color sensitive pixels 250C whenobtaining image data for use when obtaining a visual display color frameof image data. Optical reader 100 selectively reads out image data fromlight polarizing pixels 250P, or selectively extracts image datacorresponding to pixels 250P from a frame of image data when opticalreader 100 senses that a specular reflection is present or when anoperator pursuant to operator control drives optical reader 100 into areduced specular reflection read error decode mode of operation. Anoptical reader 100 including image sensor array 182H may operate inaccordance with the picture taking and decode mode flow diagram asdescribed in connection with FIG. 14 a and may execute the reducedspecular reflection read error decode mode decoding process described inconnection with FIG. 19 c.

For enhancing the performance of an optical reader according to theinvention, having an image sensor array such as image sensor array 182G,182H including light polarizing filters, optical reader 100 mayincorporate emit optics light polarizers (which may alternatively betermed “light polarizing filter elements” or “light polarizingfilters”). For example, a reader imaging module, e.g., module 1802A caninclude an optical plate 1962 as shown in FIG. 8 f which may be disposedforwardly of circuit board 1806 as shown in FIG. 8 a. Optical plate 1962can incorporate light polarizers 1963 which polarize light from lightsources 1605, 160T, that can be selectively energized when capturingimages utilizing polarizing image sensor array 182G, 182H. Lightpolarizers 1963 can be cross-polarized relative to the polarizing filterelements 261 of image sensor array 182G, 182H. Optical plate 1962 caninclude other such elements as optical diffusers (not shown) fordiffusing light rays emitted by light sources 160C-160T.

Further aspects of indicia decode circuit module 1702 andautodiscrimination circuit module 1704 are described with reference toFIGS. 22 a-22 i. Indicia decode circuit 1702 (which may be a bar codesymbol dataform decode circuit) when receiving image data transferred bycontrol circuit 552 may search the image data for markers, such as aquiet zone, indicative of the presence of a dataform, such as a one ortwo-dimensional bar code. If a potential decodable indicia (dataform) islocated, the decode circuit 1702 applies one or more indicia decodingalgorithms to the image data. If the decode attempt is successful, theoptical reader outputs decoded dataform data. All of the circuits(modules) described with reference to FIG. 22 a can be incorporated inhousing 101. Further, all of the circuits of FIG. 22 a can be embodiedby the combination of control circuit 552 and memory 566.

Optical reader 100 may also include an autodiscriminating circuit 1704.Referring to FIG. 22 a, autodiscriminating circuit 1704 may incorporatea decode circuit 1702 and an image processing and analysis circuit21208, that are in communication with one another.

As shown in this embodiment, the image processing and analysis circuit21208 comprises a feature extraction circuit 21212, a generalizedclassifier circuit 21216, a signature data processing circuit 21218, anOCR decode circuit 21222, and a graphics analysis circuit 21224 that arein communication with each other. In addition, as shown in FIG. 22 a,the feature extraction circuit 21212 comprises a binarizer circuit21226, a line thinning circuit 21228, and a convolution circuit 21230that are in communication with each other.

FIG. 22 b shows a process 21300 for employing one embodiment of theinvention utilizing the autodiscrimination circuit shown in FIG. 22 a.The process 21300 comprises an image reader recording an actuation event(step 21302), such as a receipt of a trigger signal, and in response atstep 21304, collecting (obtaining) image data from a target with theoptical reader 100. The collecting of image data step may be inaccordance with step 1204 (FIG. 14 b). After collection, the image datais transferred (step 21308) to the decode circuit 1702. The dataformdecode circuit searches (step 21310) the image data for markers, such asa quiet zone, indicative of the presence of a dataform, such as a one ortwo-dimensional bar code. If a potential dataform is located, the decodecircuit 1702 applies (step 21314) one or more dataform decodingalgorithms to the ensuing image data. If the decode attempt issuccessful, the optical reader 100 outputs (step 21318) decoded dataformdata and signals (step 21322) a successful read with an alert, such as abeep tone.

In one embodiment if the decode attempt is not successful, the imagedata is transferred (step 21326) to the image processing and analysiscircuit 21208. In another embodiment, the image data is processed inparallel with the attempt to decode the dataform data. In one suchembodiment, the process that completes first (i.e., dataform decodeattempt or the image processing) outputs its data (e.g., a decoded barcode or a captured signature) and the other parallel process isterminated. In a further embodiment, the image data is processed inresponse to the decoding of the dataform. In one such embodiment, a barcode encodes item information such as shipping label number andinformation indicating that a signature should be captured.

Within the image processing and analysis circuit 21208, the image datais processed by the feature extraction circuit 21212. In general, thefeature extraction circuit generates numeric outputs that are indicativeof the texture of the image data. As indicated above, the texture of theimage data refers to the characteristics of the type of data containedin the image data. Common types of texture include one ortwo-dimensional bar code texture, signature texture, graphics texture,typed text texture, handwritten text texture, drawing or image texture,photograph texture, and the like. Within any category of texture,sub-categories of texture are sometimes capable of being identified.

As part of the processing of the image data by the feature extractioncircuit 21212, the image data is processed (step 21328) by the binarizercircuit 21226. The binarizer circuit 21226 binarizes the grey levelimage into a binary image according to the local thresholding and targetimage size normalization. With the image data binarized, the image datais processed (step 21332) by the line thinning circuit 21228 to reducemulti-pixel thick line segments into single pixel thick lines. Withbinarized line thinned image data, the image data is processed (step21336) by the convolution circuit 21230.

In general, the convolution circuit 21230 convolves the processed imagedata with one or more detector maps designed according to the inventionto identify various textural features in the image data. In oneembodiment, the convolution circuit 21230 generates a pair of numbers,the mean and variance (or standard deviation), for each convolveddetector map. FIG. 22 c shows a set of 12 2×3 binary curvelet detectormaps 21250 used to detect curved elements present in image data. As eachof the curvelet detector maps 21250 is convolved with the image data,the mean value and the variance generated provide an indication of thepresence or density of elements in the binarized line thinned image datahaving similar shapes to the curvelet detector maps 21250. As each pixelmap generates a pair of numbers, the 12 curvelet detector maps 21250generate a total of 24 numbers. According to one embodiment, these 24numbers are representative of the curved or signature texture of theprocessed image data.

Further processing of the image data includes the outputs from thefeature extraction circuit 21212 being fed (step 21340) into thegeneralized classified circuit 21216. The generalized classifier circuit21216 uses the numbers generated by the feature extraction circuit asinputs to a neural network, a mean square error classifier or the like.These tools are used to classify the image data into general categories.In embodiments employing neural networks, different neural networkconfigurations are contemplated in accordance with the invention toachieve different operational optimizations and characteristics. In oneembodiment employing a neural network, the generalized classifiercircuit 21212 includes a 24+12+6+1=43 nodes Feedforward, BackPropagation Multilayer neural network. The input layer has 24 nodes forthe 12 pairs of mean and variance outputs generated by a convolutioncircuit 21230 employing the 12 curvelet detector maps 21250. In theneural network of this embodiment, there are two hidden layers of 12nodes and 6 nodes respectively. There is also one output node to reportthe positive or negative existence of a signature.

In another embodiment employing a neural network, the 20 curveletdetector maps 21260 shown in FIG. 22 d are used by the convolutioncircuit 21230. As shown, the 20 curvelet detector maps 21260 include theoriginal 12 curvelet detector maps 21250 of FIG. 22 c. The additional 8pixel maps 21260 are used to provide orientation information regardingthe signature. In one embodiment employing the 20 curvelet detector maps21260, the generalized classifier circuit 21212 is a 40+40+20+9=109nodes Feedforward, Back Propagation Multiplayer neural network. Theinput layer has 40 nodes for the 20 pairs of mean and variance outputsgenerated by a convolution circuit 21230 employing the 20 curveletdetector maps 21260. In the neural network of this embodiment, there aretwo hidden layers of 40 nodes and 20 nodes respectively, one output nodeto report the positive or negative existence of a signature, and 8output nodes to report the degree of orientation of the signature. Theeight output nodes provide 2⁸=256 possible orientation states.Therefore, the orientation angle is given in degrees between 0 and 360in increments of 1.4 degrees.

In some embodiments, the generalized classifier circuit 21216 is capableof classifying data into an expanded collection of categories. Forexample, in some embodiments the generalized classifier circuit 21216specifies whether the image data contains various data types such as asignature; a dataform; handwritten text; typed text; machine readabletext; OCR data; graphics; pictures; images; forms such as shippingmanifest, bill of lading, ID cards, and the like; fingerprints,biometrics such as fingerprints, facial images, retinal scans and thelike, and/or other types of identifiers. In further additionalembodiments, the generalized classifier circuit 21216 specifies whetherthe image data includes various combinations of these data types. Insome embodiments, the general classifier circuit 21216 specifies whetherthe image data contains a specified type of data or not. In one suchembodiment the image processing and analysis circuit 21208 is containedwithin an identification circuit that outputs an affirmative or negativeresponse depending on the presence or absence of the specified datatype, such as a signature or a biometric in the image data.

In one embodiment once the presence of a signature has been confirmedand its general orientation determined, image data is transferred (step21344) to the signature data processing circuit 21218. In oneembodiment, the signature data processing circuit 21218 is used todetect the boundaries of the signature in the image data. In oneembodiment, the signature boundary is detected using a histogramanalysis. As shown in FIG. 22 e, a histogram analysis consists of aseries of one-dimensional slices along horizontal and verticaldirections defined relative to the orientation of the signature. In oneembodiment, the value for each one-dimensional slice corresponds to thenumber of black (i.e., zero valued) pixels along that pixel slice. Insome embodiments if no bar codes have been decoded, then some specifiedregion of the full frame of image data, such as a central region iscaptured for signature analysis. Once completed, the histogram analysisprovides a two-dimensional plot of the density of data element pixels inthe image data. The boundary of the signature is determined with respectto a minimum density that must be achieved for a certain number ofsequential slices. In one embodiment, the histogram analysis searchesinwardly along both horizontal and vertical directions until the pixeldensity rises above a predefined cutoff threshold. So that the signaturedata is not inadvertently cropped, it is common to use low cutoffthreshold values.

In one embodiment, once the boundaries of the signature have beendetermined, the signature data processing circuit 21218 crops the imagedata and extracts the signature image data. In one such embodiment, thecropping is performed by an image modification circuit that generatesmodified image data in which a portion of the image data not includingthe signature has been deleted. In other embodiments, variouscompression techniques are employed to reduce the memory requirementsfor the signature image data. One such technique includes the encodingof the signature image data by run length encoding. According to thistechnique, the length of each run of similar binarized values (i.e., thelength of each run of 1 or 0) for each scan line is recorded as a meansof reconstructing a bit map. Another encoding technique treats thesignature image data as a data structure where the elements of the datastructure consist of vectors. According this encoding technique, thesignature is broken down into a collection of vectors. The position ofeach vector in combination with the length and orientation of eachvector is used to reconstruct the original signature. In one suchembodiment, the encoding process generates a new vector whenever thecurvature for a continuous pixel run exceeds a specified value. Afurther compression technique employs B-Spline curve fitting. Thistechnique has the capacity to robustly accommodate curvature and scalingissues.

In various embodiments, the signature image data or a compressed orencoded version of the signature image data is stored locally on adedicated memory device. In one such embodiment, the local memory devicecan be a detachable memory device such as a CompactFlash memory card orthe like described in more detail below. In another embodiment, thesignature image data is stored in a volatile or non-volatile portion ofgeneral purpose memory and downloaded at a future time. In a furtherembodiment, the signature image data can be transmitted via wired orwireless means either at the time of capture or at a later point, suchas when a data collection session has been completed.

In another embodiment, the signature data processing circuit 21218 doesnot perform a histogram analysis but simply stores in memory the entireimage or a compressed version once the presence of a signature has beendetermined. In a further embodiment to save processing time, the initialimage analysis is performed on a lower resolution image. Once thepresence of a signature is determined in this embodiment, a higherresolution image is taken. In one embodiment, a signature extractionhistogram analysis is performed on this image. Next, the image is storedin memory in either compressed or original format. In some embodiments,the image data is combined with other data to form a record for aparticular item such as a package or shipping envelope. As mentionedabove, some of the additional data that can be collected by the opticalreader 100 and stored with or separate from the signature data includesbut is not limited to dataform data, handwritten text data, typed textdata, graphics data, image or picture data, and the like.

As part of its operations, the image processing and analysis circuit21208 can be designed to perform specialized tasks for different datatypes. For example, if the generalized classifier circuit 21216determines that the image data contains typed or machine readable text,the image data can be collected, possibly histogram analyzed, and storedor alternatively, the image data can be transferred to the OCR decodingcircuit 21222. Similarly, if the generalized classifier circuit 21216determines that the image data includes a graphic element, the imagedata can be transferred to the graphics analysis circuit 21224 forprocessing. In one embodiment, the graphics analysis circuit 21224 isconfigured to recognize and decode predefined graphics. In one suchembodiment, the graphics analysis can include determining which, if any,boxes have been selected in the billing and shipping instructions on ashipping label. In a further embodiment, the graphics analysis caninclude locating and decoding the typed or handwritten text contained inthe zip code box on a shipping label. In an alternative embodiment, theoptical reader 100 can be configured to automatically attempt decodeoperations in addition to the dataform decode, such as OCR decoding orgraphics decoding, prior to the activation of the feature extractioncircuit 21212.

In another embodiment, the image processing and analysis circuit 21208segments the image data into regions and performs a feature extractionand general classification analysis on each region. In one embodiment asshown in FIG. 22 f, the standard rectangular image data window isdivided into four equal sized sub-rectangles. In another embodimentshown in FIG. 22 g, the segmentation consists of overlapping regions sothat the total area of the segmented regions is larger than that of thecomplete field of the image data. In FIG. 22 g there are seven shownoverlapping regions where each identifying numeral is shown in thecenter of its region. In a further embodiment shown in FIGS. 22 h and 22i, the segmentation consists of sample regions (shown as cross-hatched)within the complete field of the image data. In another embodiment, thesampled regions can be based on a preloaded user template that, forexample, identifies regions of interest such as a signature regionand/or a bar code region, in for example, a shipping label.

In one embodiment, the segmentation process is used to identify thelocation of a signature in image data the might include additionalelements such as dataforms including bar code dataforms, text, graphics,images and the like. In one such embodiment the generalized classifiercircuit 21216 classifies the contents of each region of the segmentedimage data. The region containing the signature is then extracted by thesignature data processing circuit 21218. In one embodiment if multipleregions are indicated as containing signature data, the signature dataprocessing circuit 21218 analyzes the arrangement of these regions toidentify the region most likely to contain the image data. In a furtherembodiment when multiple regions are indicated as containing signaturedata, the image processing and analysis circuit 21208 establishes afeedback loop where additional segmented regions are generated andanalyzed until a single segmented region containing signature data islocated.

Additional image processing operations which may be carried out byoptical reader 100 are described in U.S. patent application Ser. No.10/958,779, filed Oct. 5, 2004 entitled, “System And Method ToAutomatically Discriminate Between A Signature And A Barcode” andincorporated herein by reference in its entirety.

Various applications which may be carried out by any of the opticalreaders 100 that have been described herein have been described withreference to FIGS. 10, 11, 12 a and 12 b. Another application which canbe carried out with any optical reader 100 described herein is describedwith reference to FIGS. 13 a-13 e. In FIG. 13 a a motor vehicle 1282 isshown which may be a delivery vehicle or a passenger vehicle. Vehicle1282 has a license plate 1314, a vehicle identification number (VIN)sticker 1306, typically located on the driver's side door jam. The VINsticker 1306 carries a printed VIN number 1308 and a bar code symbol1310. A VIN number is an alphanumeric unique vehicle identificationnumber assigned at the time of manufacture of the vehicle. Vehicle 1282may further include a VIN plate 1314 (FIG. 13 c) carrying the charactersof the VIN number etched on a metal plate and located under the vehiclewindshield 1351, and a vehicle registration sticker 1320. Vehicle 1282has a plurality of machine readable vehicle identifiers. Specifically,the characters of license plate 1284 can be OCR decoded by opticalreader. Further, VIN sticker 1308 has a VIN bar code 1310 andregistration sticker 1320 which may include a plurality of bar codesymbols 1322, 1324 encoding the vehicle registration number and possiblyredundantly encoding the VIN number of vehicle 1282. The charactersetched on VIN plate 1314 can also be subject to OCR decoding by opticalreader 100. Further, the VIN characters of VIN sticker 1306 can besubject to OCR decoding by optical reader 100. It may be advantageous toutilize an optical reader including light polarizing pixels 250P havinglight polarizing filter elements 261 when reading VIN plate 1314 giventhat specular reflection read conditions are more prevalent whendecoding indicia encoded by etching on metal surface.

In an application for utilizing optical reader 100 relative to vehicle1282, several identifiers of vehicle 1282 may be decoded and severalcolor pictures of vehicle 1282 may be taken. The decoded message datatogether with the color picture data may then be uploaded to a remoteserver 184 (FIG. 10) which archives and creates accessible web pagescontaining reports summarizing the identifier and picture information.In one application LAN 170 (FIG. 10) is a LAN at an automobile insuranceclaim center, LAN 185 is a distant data archiving center operated by theautomobile insurance provider and LAN 2170 is a LAN apart from LAN 170and LAN 185 and may be located, e.g., at a claim center of the insuranceprovider other than the claim center in which LAN 170 is located.

Optical reader 100 may be configured so that when an operator actuates adesignated user interface control button such as button 3158 (FIG. 9 b)an auto insurance application form 1362 is displayed on display 504which aids an operator of optical reader 100 in entering data intoreader 100. Form 1362 first prompts an operator to read several machinereadable identifiers of vehicle 1282. Form 1362 prompts an operator toread VIN bar code symbol 1310, then characters of VIN plate 1314, thenthe first registration sticker bar code symbol 1310, then the secondregistration sticker bar code symbol 1324, then the character of thelicense plate 1284. The text corresponding to each identifier may behighlighted when data corresponding to the identifier is read. When datacorresponding to identifier decode section 1363 of form 1362 is beingentered, optical reader 100 is in a decode mode of operation such thatactuation of trigger 216 causes optical reader 100 to obtain a decodeframe at step 1204 and transfer the decode frame to decode circuit 1702.The decode frame may contain monochrome image data read from a hybridmonochrome image sensor array 182, 182A. Where optical reader 100 hasseparate picture taking and decoding imaging assemblies as described inconnection with FIGS. 17 a-17 g, the decode frame at step 1204 isobtained by actuation of the imaging assembly within block 598 (FIG. 17a). When entry of decoded vehicle identifier information is complete, anoperator toggles to line 1365 and clicks an appropriate key of keyboard508 to indicate that identifier decoding is complete. Form 1362 thenprompts an operator to take pictures of vehicle 1282 for purposes ofmaking a record of the damage to vehicle 1282. The inventor discoveredthat the incorporation of color filter elements into an image sensorarray 182 of optical reader 100 facilitates the obtaining of visualdisplay frames of image data that accurately record damage to a vehicle.With visual display color frames of image data corresponding to vehicle1282 being stored and/or displayed for visual display, damage to vehicle1282 can readily be assessed by visual inspection of the visual displayframes when displayed on a display 504, 1504. When damage records arerecorded with color image data, the amount of paint scratched from avehicle, for example, can readily be assessed by visual inspection.Section 1364 of display form 1362 prompts an operator to take severalcolor pictures of vehicle. When picture entry section 1364 of form 1362is being executed, optical reader 100 is in a picture taking mode suchthat actuation of trigger 216 causes a visual display frame of imagedata to be obtained at step 1404 (FIG. 14 c). The visual display frameof image data may be output to e.g., a storage device and/or a displaydevice. When data corresponding to form section 1364 is being entered,an operator may use optical reader 100 to take several color pictures ofdamaged area 1370 of vehicle 1282. While executing obtain step 1404,control circuit 552 may selectively read out color image data from colorsensitive pixels 250C as described herein and possibly utilizemonochrome image data for enhancement of the information content of thecolor image data. Where optical reader 100 includes a pair of imagingassemblies as described in connection with FIGS. 17 a-17 g, controlcircuit 552 at step 1404 may actuate color image sensor array 182D forexecution of obtain step 1404. When an operator inputs a confirmationthat all necessary pictures of vehicle 1282 have been taken by togglingto line 1367 and clicking an appropriate key of keyboard 508, controlcircuit 552, which may be incorporated in hand held housing 101, mayformat obtained visual display color frames of image data in one or moresuitably image file formats, (e.g., .BMP, .TIFF, .PDF, .JPG, .GIF)assemble all the collected decoded vehicle identifier data and all ofthe visual display color frames of image data corresponding to vehicle1282 into a transaction data set, and send the transaction data set todistant remote server 184. Control circuit 552 may date/time stamp thetransaction data set on sending. The File Transfer Protocol (FTP) may beutilized to send the transaction data set or another suitable filetransferring protocol configured to carry associated decoded vehicleidentifier data (such as decoded VIN bar code data and decode vehicleregistration bar code data) and color image data. Server 184 may storethe received transaction data set into a database as indicated bydatabase 187 including similar information from other vehicles at otherclaim centers. Server 184 may be configured to create viewable web pagessummarizing the transaction set data (e.g., the date/time stampedcombined VIN, registration number, license plate number and record-ofdamage visual display color frames of image data). These web pages maybe viewed using any PC in communication with IP network, e.g., PC 172and PC 2172.

While the present invention has necessarily been described withreference to a number of specific embodiments, it will be understoodthat the time, spirit, and scope of the present invention should bedetermined only with reference to the following claims:

1. A method comprising: providing an optical reading device, wherein theproviding includes providing the optical reading device to include animage sensor array and an imaging lens, wherein the image sensor arrayincludes a first subset of pixels and a second subset of pixels, thefirst subset of pixels being monochrome pixels without color filterelements, the second subset of pixels being color pixels havingwavelength selective color filter elements, wherein a pixel count of thefirst subset of pixels is larger than a pixel count of the second subsetof pixels, wherein pixels of the first subset of pixels have higherlight transmissivity than pixels of the first subset of pixels;generating monochrome image data and color filter array image datautilizing the image sensor array, the monochrome image data having afirst resolution, the color filter array image data having a secondresolution, the first resolution being greater than the secondresolution. processing color filter array image data for output of acolor frame of image data having a resolution greater than the secondresolution, wherein the processing includes utilizing monochromeintensity information of the monochrome image data; outputting a colorframe of image data, the color frame of image data having a resolutiongreater than the second resolution.
 2. The method of claim 1, whereinthe providing the optical reading device includes providing the imagesensor array so that the pixels of the second subset of pixels areuniformly or substantially uniformly distributed throughout the imagesensor array.
 3. The method of claim 1, wherein the providing theoptical reading device includes providing the image sensor array so thata pixel of the second subset of pixels has eight adjacent monochromepixels.
 4. The method of claim 1, wherein the providing the opticalreading device includes providing the image sensor array to include red,green and blue pixels.
 5. The method of claim 1, wherein the providingthe optical reading device includes providing the image sensor array toinclude magenta and cyan pixels.
 6. The method of claim 1, wherein theprocessing includes de-mosaicing the color filter array image data forproviding of a de-mosaicized color image.
 7. The method of claim 1,wherein the processing includes processing the color filter array imagedata so that the color frame of image data for output includes pixelpositions corresponding to each pixel of the image sensor array.
 8. Themethod of claim 1, wherein the processing includes determining a colorintensity value for pixel positions of a color image.
 9. The method ofclaim 1, wherein the processing includes determining a color intensityvalue for pixel positions of a color image utilizing color values of aplurality of colors by calculating a weighted sum of a plurality ofcolor values, wherein the plurality of color values are weightedunequally.
 10. The method of claim 1, wherein the second subset ofpixels includes red, green and blue pixels and wherein the processingincludes determining a color intensity value for a pixel position of acolor image utilizing red, green and blue color values.
 11. The methodof claim 1, wherein the second subset of pixels includes red, green andblue pixels and wherein the processing includes determining a colorintensity value for a pixel position of a color image, the determiningincluding calculating a weighted sum of red, green and blue colorvalues, wherein the red, green and blue color values are weightedunequally.
 12. The method of claim 1, wherein the outputting includesoutputting a visual display frame of image data having a resolutionequal to a pixel resolution of the image sensor array.
 13. The method ofclaim 1, wherein the outputting includes outputting a visual displaycolor frame of image data to a display.
 14. The method of claim 1,wherein the outputting includes outputting a visual display color frameof image data to memory.
 15. The method of claim 1, wherein thegenerating includes reading out a single frame of image data from theimage sensor array.
 16. The method of claim 1, wherein the generatingincludes reading out a plurality of frames of image data from the imagesensor array.
 17. The method of claim 1, wherein the utilizingmonochrome intensity information includes calculating for a common pixelposition of a monochrome image and a color image an intensity valuedelta between a monochrome intensity value and a color intensity valuefor the common pixel position.
 18. The method of claim 1, wherein theutilizing monochrome intensity information includes calculating for acommon pixel position of a monochrome image and a color image anintensity value delta between a monochrome intensity value and a colorintensity value for the common pixel position, and updating color valuesof the color image for the common pixel position utilizing the intensityvalue delta.